Skip to main content
2024 BMES Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Bioinformatics, Computational and Systems Biology

Bioinformatics, Computational and Systems Biology - Poster Session C

Poster A9 - Joint Sequence and Structure Embeddings of Spike Proteins for Predicting Viral Dynamics

Friday, October 25, 2024
10:00 AM - 11:00 AM EST
Location: Exhibit Hall E, F & G

Introduction:  Predicting which viral strains will become predominant in human populations is challenging due to the complex interplay of viral protein structures and sequences with factors such as binding affinity and immune escape. A comprehensive understanding and anticipation of viral evolution is crucial for effective vaccine design and therapeutic interventions. Traditional methods in virology have been limited by considering sequence or structural data in isolation. Addressing this limitation, our study introduces ViSENet (Viral Sequence Evolution Network), a model that learns joint representations from both protein structure and sequence data—along with key properties like binding affinity and immune escape—to create a more holistic model of viral evolution. ViSENet employs a deep transformer-based autoencoder that not only generates sequence embeddings that are temporally organized but also infers the chronological emergence of these sequences. This is complemented by a geometric scattering autoencoder that captures structural embeddings from AlphaFold-predicted protein structures. By combining these two data representations in a joint latent space, our model surpasses the predictive performance of previous, unimodal methods. Furthermore, by utilizing the neural ODE framework to navigate the joint latent space, our model can project viral evolution dynamics several weeks into the future. Such forecasting capabilities are demonstrated through accurate predictions of SARS-CoV-2 and influenza evolutionary patterns, showcasing the potential of our approach in guiding future vaccine and therapeutic development.

Materials and

Methods:  The proposed architecture, ViSENet, is designed to model multiple properties of the SARS-CoV-2 spike protein sequence through a multi-task objective. The architecture uses a multi-task loss formulation that organizes the latent space by sequence, structure, and fitness while simplifying the search for a high-fitness spike protein. ViSENet utilizes a transformer encoder to learn the mapping between input sequence to a point in latent space that reflects sequence information, structure, and properties included in the multi-task objective. As observed in previous works, the inclusion of an auxiliary supervised task during training results in embedding organized by the auxiliary labels.

The sequence model uses a straightforward tokenization strategy whereby each amino acid is mapped to an unique integer. The encoding of the spike protein sequence is then passed to an attention-based pooling layer that calculates the convex sum. Dimensionality is further reduced using a fully connected feed-forward network.

The structure model uses AlphaFold2-generated structures of the spike-protein Receptor Binding Domain (RBD). It is known that the RBD highly impacts viral entry. Therefore, using the RBD accelerates the folding process while providing more salient information to the model. We then build a graph from the tertiary structure and use geometric scattering to capture meaningful information. This structure embedding is then appended to the sequence and they are used as the inputs for our encoder. Figure 1 shows a schematic of the complete model architecture.

Results, Conclusions, and Discussions: Visualization of latent representations obtained by ViSENet revealed that embeddings were organized by (1) WHO variant and lineage, (2) time (date of emergence), and (3) nucleotide edit distance between sequences (Figure 2). We trained a neural ODE in this highly-structured latent space to predict viral dynamics. We also forecasted viral evolution and compared the binding affinity and amino acid composition of the predicted sequences against withheld data. As shown in Tables 1 and 2, ViSENet achieved state-of-the-art performance with significantly lower error compared to other knowledge graphs (Otter) and protein language models (ProstT5).

We also performed ablation studies: we trained the model using only sequence information and only structure information. Although the structure-only model implicitly contains sequence information, we found that using both inputs improved model performance.

Therefore, for future works, we intend in performing a shared attention mechanism between sequence and structure. Instead of generating and attending to two different embeddings, we can build a shared attention for which the sequence and structure weights are identical. We expect this methodology to further improve latent space organization because there is no difference in information between amino acids on the primary and tertiary structures.

Another intended iteration is analysis of new variants, specifically using the old datasets we had to show the possibility of prediction for the new variants (for example FLIRT SARS-CoV-2 strains that are subvariants of Omicron).

Acknowledgements (Optional):  I wish to thank the YCDO and the Hahn Scholars Program for financial support.