Poster L15 - Incorporating bounded data for classification of molecular potency improvements

Introduction: During drug design, machine learning workflows can screen vast molecular spaces to efficiently triage experiments. Unfortunately, a substantial amount of available data remains inaccessible to modern machine learning algorithms. For example, a fifth of publicly available IC50 values is reported as bounded values, limiting regression algorithms due to their inability to use this data. We previously showed that leveraging pairwise molecular representations as training data can improve regression prediction of molecular properties. Here, we extend this approach into a novel classification task wherein algorithms are tasked to predict which of two paired molecules are more potent, granting access to bounded datapoints. We evaluate the ability of established classification algorithms to learn from this paired data (“DeltaClassifiers”) and compare their ability to correctly compare potencies of two molecules with three established, state-of-the-art regression algorithms (Random Forest, XGBoost, and Chemprop). Across 230 ChEMBL IC50 datasets, both tree-based and neural network-based DeltaClassifiers outperform traditional regression approaches when classifying potency improvements, showcasing their potential to guide optimization campaigns for drug discovery and other material design tasks.

Materials and

Methods: ChEMBL33 was curated to generate 230 benchmarking IC50 datasets. For our deep DeltaClassifier, we used the established, two-molecule version of Chemprop and for our tree-based DeltaClassifier we used XGBoost as base models. For standard regression approaches to classify potency improvements, machine learning models were trained on absolute IC50 values, and these models were used to predict absolute IC50 values for new molecules that were subsequently subtracted to classify potency differences based on the sign of this difference. For DeltaClassifier models, the training data was first cross-merged into all possible molecule pairs and a ground-truth classification label was created using the sign of the subtracted ground-truth IC50 values. Additionally, during “demilitarization”, any pairs with a ground-truth IC50 difference below the demilitarization threshold value or with unknown potency differences were removed prior to training the model. For Chemprop-based models, molecules were described using atom and bond features. For Random Forest and XGBoost-based models, individual molecules were described using radial chemical fingerprints. Models were evaluated using random 3x10-fold cross-validation and scaffold-split (deepchem) with metrics of accuracy, F1 score, and ROCAUC. To prevent data leakage, training data was first split into train and test sets prior to cross-merging to create molecule pairings so no molecule will be present in pairs made in both the training and test set.

Results, Conclusions, and Discussions: The deep learning-based DeltaClassifer showed good performance with ROC-AUC values up to 0.98 and never below 0.68. When comparing the performance to other established state-of-the-art algorithms, we noticed that the DeltaClassifier outcompeted all regression methods in at least 69% of datasets and was competitive in at least 96% of datasets for all metrics, ranking highest on average (1.3, accuracy) which led to a median, modified Z score across all benchmarking datasets of 1.0 in terms of accuracy with similar results for other metrics. Significantly, our tool also outperformed all state-of-the-art approaches when predicting molecules with shared or distinct scaffolds, highlighting its utility for novel material design and fine-tuned optimization. Our tool’s performance was strong independent of the investigated drug target, with improved performance across all EC classes of protein targets. Taken together, DeltaClassifiers and subsequent extensions stand to accurately guide potency optimizations by integrating all available data. Beyond drug development, DeltaClassifier can be implemented for material selection and optimization to de-risk these endeavors.

Acknowledgements (Optional): We thank Pat Walters for insightful technical discussions about this work. We would also like to thank the Chemprop and the Scikit-learn developers for making their machine learning algorithms publicly available. Z.F. is supported by the Department of Defense (DoD) through the National Defense Science & Engineering Graduate (NDSEG) Fellowship Program. This research was supported by the Duke Science & Technology Initiative and by the NIH NIGMS grant R35GM151255.