Skip to main content
2024 BMES Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Bioinformatics, Computational and Systems Biology

Bioinformatics, Computational and Systems Biology - Poster Session C

Poster A1 - Exploring the use of large language model embeddings for stroke medication recommendations

Friday, October 25, 2024
10:00 AM - 11:00 AM EST
Location: Exhibit Hall E, F & G

Introduction:  Artificial intelligence (AI) and machine learning (ML) can provide significant benefits to the healthcare system. When time is of the essence, AI/ML systems can assist providers with treatment plans, recommendations, and improve time and quality of care. Dr. Ceskoutsé et al. display this kind of interaction with their CRGANNC system, which they cover in “Sub-clustering based recommendation system for stroke patient: Identification of a specific drug class for a given patient.” Their work shows that machine learning can be used to recommend stroke treatments in a timely manner and open the door for further applications. In this work, we explore the use of large language model to generate embedding vectors that would capture the essential representation of patient symptoms and use a ML model to make predictions on those systems.

Materials and

Methods:  We utilized the dataset prepared by Ceskoutsé et al., (Ceskoutsé et al., 2024) that contains the patient symptoms, demographics, and drug recommendation in tabular format. Kaggle Stroke disease dataset sourced from Federico Soriano was preprocessed and augmented with the help of tabular Generative Adversial Network that extended original dataset with synthetic data. Feature engineering was done to create extended information including the identification of four classes of drugs: none (no medication), anticoagulant, antihypertensive, and fibrate. The dataset's distribution of drug recommendations was predominantly 'none' with 8671 cases, followed by 'fibrate' with 585, 'antihypertensive' with 292, and 'anticoagulant' with 143 cases.

These processed datasets were then split into the ratio of 70:20:10 as a training dataset, an evaluation dataset, and a test dataset. The test dataset was set aside during the model training process. From the rest of the dataset splits, we formulated a prompt for each observation of data. Then we submitted this prompt to the large language model named “distilBERT-base-uncased" to generate embedding. These embedding vectors were finally used to train XGBoost model with the objective function of “multi:softmax” and evaluation metric of “mlogloss” to learn and provide medication recommendations for patients based on their demographics and symptoms.

Results, Conclusions, and Discussions: Regarding the analysis of the training loss curve, we observed that the loss was smooth and decreased over boosting round indicating that the model was able to learn the patient symptoms and make predictions. The performance of the classification model on unseen data was quantitatively assessed through several metrics: accuracy, precision, recall, and F1-score, with a focus on class-specific performance. The model achieved an overall accuracy of 99%. Class “anticoagulant” showed a precision of 0.43 but a perfect recall of 1.00, resulting in an F1-score of 0.60. Classes “antihypertensive,” “fibrate,” and “none (no medication)” demonstrated high precision and recall values. Both Class “antihypertensive” and Class “none (no medication)” achieved perfect scores of 1.00 for both metrics, while Class “fibrate” showed a slightly lower recall at 0.89.
The approach of transforming patient symptoms into embedding captured nuanced features, yet the evaluation for minority classes was limited due to the class imbalance. Future work could include exploring class balancing techniques for better results. The model would eventually allow healthcare staff to input symptoms and receive drug recommendations on stroke assisting in decision-making.
In this study, we extracted embedding vectors using a large language model of the patient symptoms enhancing the representational capacity of the features. Then we trained XGBoost model on these vectors which yielded promising results on drug recommendations in stroke patients. However, further data augmentation for minority classes must be performed for enhanced precision.

Source:
Ceskoutsé, R. F., Bomgni, A. B., Gnimpieba Zanfack, D. R., Agany, D. D. M., Bouetou Bouetou, T., & Gnimpieba Zohim, E. (2024). Sub-clustering based recommendation system for stroke patient: Identification of a specific drug class for a given patient. Computers in Biology and Medicine, 171, 108117. https://doi.org/10.1016/j.compbiomed.2024.108117

Acknowledgements (Optional):