Skip to main content
2024 BMES Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Bioinformatics, Computational and Systems Biology

Bioinformatics, Computational and Systems Biology - Poster Session E

Poster L12 - Deep Learning for 3D Protein Structure Prediction from AFM Images

Saturday, October 26, 2024
10:00 AM - 11:00 AM EST
Location: Exhibit Hall E, F & G

Introduction:  Recent progress in deep learning has significantly advanced the prediction of 3D protein structures, particularly by using protein sequences and Cryo-Electron microscopy (Cryo-EM) images as inputs. However, these methods often underperform in predicting the structures of protein complexes (PCs), which consist of multiple proteins. Our research explores the combination of atomic force microscopy (AFM) with deep learning to predict the 3D structures of PCs. AFM provides height maps that illustrate PCs in various random orientations, offering valuable data for training a neural network to predict their 3D structures. We utilize the pre-trained UpFusion model, which incorporates a conditional diffusion model for generating novel views, to train an instance-specific NeRF model for 3D reconstruction. We assess UpFusion's effectiveness through zero-shot predictions of 3D protein structures from AFM images. Collecting actual AFM images poses a significant challenge due to its time-intensive nature. To overcome this, we employ a virtual AFM imaging technique that converts `PDB' protein files into multi-view 2D virtual AFM images using volume rendering. We conduct extensive validation of the UpFusion architecture with both virtual and real multi-view AFM images. Our findings include comparisons of predicted structures using different numbers and combinations of views. This innovative approach promises substantial improvements in the accuracy of predicting protein complex structures with additional refinements to the UpFusion network.

Materials and

Methods:  We utilize UpFusion, a pre-trained neural network, for synthesizing novel views and inferring 3D representations from sparse image sets without corresponding pose data. This strategy diverges from traditional sparse-view 3D inference methods that rely on precise camera poses, which are often unavailable in real-world scenarios. UpFusion leverages a conditional generative model that uses available images as context, eliminating the need for explicit camera poses.

The core methodology of UpFusion combines the Unposed Scene Representation Transformer (UpSRT) and a denoising diffusion model. UpSRT uses input images without pose information to infer features for query views, which then conditions the diffusion model. This setup facilitates novel view synthesis from sparse data and includes 'shortcuts' via attention mechanisms for direct feature access during generation.
Although the conditional diffusion model produces high-quality renderings, maintaining 3D consistency across views remains challenging. To address this, the process is refined by optimizing an instance-specific neural representation. Inspired by SparseFusion, this refinement enhances the likelihood of accurate 3D renderings using a Score Distillation Sampling (SDS) loss adapted for view-conditioned generative models.

Training UpFusion involves multiple stages: initially training the UpSRT with a reconstruction loss on predicted colors for queried rays from reference images, followed by conditioning the diffusion model with UpSRT outputs. Finally, a Neural Radiance Field (NeRF) is optimized over 3000 iterations, requiring about an hour on an A100 GPU, highlighting the effective integration of transformers and diffusion models for 3D inference and novel view synthesis.

Results, Conclusions, and Discussions: We assessed the zero-shot 3D reconstruction capabilities of the UpFusion network using both virtual and actual unposed AFM images. We explored the impact of different numbers of input views (e.g., 1V, 3V, and 6V) on reconstruction quality, noting improvements with more images. To quantify this, we used metrics such as Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS). A numerical analysis across four randomly selected protein samples yielded average values with 6 views showing a PSNR of 16.72 dB (consider as fair value), SSIM of 0.8737 (consider as good value), and an LPIPS of 0.1677 (consider as good value). These results align with visual predictions, showing that more input views enhance view synthesis quality.

Additionally, we investigated how the diversity of input views affects prediction accuracy. We found that a varied set of views generally improves 3D reconstructions and reduces ambiguities compared to similar-looking views. We visualized predictions using actual AFM images of the WRC protein complex and compared them with those from virtual AFM images. Both captured the high-level shape, but actual AFM images missed finer details. This was expected as these were zero-shot predictions from a network untrained on AFM or protein datasets. The main challenge noted was the discrepancy in depth information interpretation, as the color in AFM images represents depth, unlike the typical training data involving real-life objects with standard lighting.

In this study, we validated the UpFusion deep learning framework, designed for generating novel views and 3D NeRF reconstructions, for protein structure prediction using AFM images. We employed a virtual AFM imaging pipeline, a GPU-accelerated volume rendering technique, to create virtual AFM-like images from `PDB' files for extensive evaluation. We conducted zero-shot predictions on both actual and virtual AFM images without model fine-tuning. The network showed promising potential in accurately predicting 3D structures. Future work will refine UpFusion specifically for AFM-based protein structure predictions, with extensive fine-tuning expected to significantly enhance 3D reconstruction accuracy.

Acknowledgements (Optional):  This work was partly supported by the seed grant from Translational AI Center (TrAC) at Iowa State University.