Poster M7 - TRACER—Transformers with Multi-head Attention and Coarse-grain Analysis for TCR-pMHC Recognition

Introduction: Forecasting interactions between T-cell receptors (TCR) and peptides is crucial for understanding adaptive immunity and enhancing immunotherapeutic interventions. A primary challenge in determining TCR specificity is the vast array of potential TCR and antigen combinations, in contrast to the limited number analyzable per individual. Recent advancements have underscored the benefits of integrating structural data to reduce the extensive need for sequence training data, yet decoding the intricate TCR-antigen structural interface to predict MHC-allele-restricted TCR-peptide interactions remains arduous. We have developed a coarse-grained structural template model that leverages TCR-peptide spatial interactions from publicly available TCR-antigen crystal structures. This model efficiently identifies unknown TCRs and their corresponding peptides in our collection, albeit with suboptimal performance in cases lacking crystal structures. To enhance our predictions, we introduced the TRACER model, a multi-head transformer architecture. The first module processes peptide sequences and CDR3 sequences from TCR\(\beta\) and TCR\(\alpha\) chains using an attention mechanism to highlight amino acid interactions and learn local sequence features. It employs cross-attention to capture interactions between TCR and peptide sequences, while the second module integrates spatial data from crystal structures, improving learning through our model’s attention mechanisms. Additionally, to tackle challenges with unstable TCR-pMHC pairs, we incorporated the on-rate constant \(k_{\text{on}}\) into our model, significantly enhancing the prediction accuracy for these difficult cases. This approach, merging biophysical, kinetic, and attention-based strategies, broadens the applicability in studies of structurally and sequentially diverse TCR-peptide pairs, forecasting TCR-peptide bindings more accurately.

Materials and

Methods: We developed the TRACER model, which utilizes a multi-head Transformers architecture alongside a coarse-grained model (RACER) to predict TCR-pMHC binding affinity. The model inputs include sequences of TCR$\alpha$, TCR$\beta$, and peptides, combined with 3D structural data, all processed and encoded for detailed biochemical representation. The architecture of the model comprises three principal modules: (1) Feature Extraction, which employs multi-head self-attention mechanisms to extract both local and global features of the sequences and their corresponding 3D structures; (2) Feature Fusion, utilizing a cross-attention mechanism that integrates features from TCRs and peptides. This module focuses on interaction-relevant amino acids and their biophysical distances, enhanced by insights from the RACER model, to improve interpretability; (3) Classification, where a fully-connected neural network determines the binding probabilities. For certain unstable TCR-pMHC pairs, such as NY-ESO, where traditional sequence and 3D structural data were insufficient, we incorporated kinetic characteristics into the model. Specifically, the inclusion of the on-rate constant $k_{\text{on}}$ proved to be a critical enhancement, enabling accurate predictions for these challenging cases. The model's performance was evaluated using metrics such as accuracy, specificity, sensitivity, and area under the curve (AUC). During the training phase, we utilized the VDJdb dataset for sequence data and the ATLAS database for crystal structures. The testing phase employed experimental data from MD Anderson Cancer Center (MDACC), comprising a general dataset for MART1, FLU, and NLV cases, alongside a specialized dataset that included kinetic features aimed at enhancing predictions for unstable TCR-pMHC interactions, such as those involving NY-ESO.

Results, Conclusions, and Discussions: In our study, the TRACER model demonstrated improved performance in predicting TCR-pMHC binding(2), and excelled in classifying previously unseen peptides. Through cross-validation conducted with three different sets, the structure-based model and TRACER together outperform other traditional models, as evidenced by the AUROC for each peptide (Figure 1). This underscores its efficacy despite performance variability across different peptides. Moreover, the challenges associated with predicting binding affinity for unstable peptides like NY-ESO highlight inherent limitations within current modeling techniques. Our findings, particularly those related to the low correlation between dissociation constants ($K_d$) and off-rates ($k_{\text{off}}$) as shown in Figure 2, suggest that traditional kinetic parameters may not suffice in all scenarios. Furthermore, the observed variability in on-rates ($k_{\text{on}}$) for cases with high $K_d$, as presented in Figure 2, reinforces the need for integrating a broader spectrum of kinetic parameters into our models. Looking forward, the incorporation of comprehensive kinetic properties, including $k_{\text{on}}$, into the TRACER model is expected to future enhance its predictive accuracy and applicability.

In summary, our combined TRACER modeling framework shows encouraging promise for leveraging sequence and structural features together to predict TCR-pMHC binding, particularly distinguishing itself in handling peptides not previously encountered during the training phase. In applications to independent dataset the model performs well relative to alternative models.

We anticipate that such hybrid modeling schemes can offer flexible prediction of TCR-pMHC specificity in applications for optimizing adaptive T cell immunology.

Acknowledgements (Optional): This work was supported by the Cancer Prevention and Research Institute of Texas (CPRIT RR210080). JTG is a CPRIT Scholar in Cancer Research.