Poster S2 - Deep learning-based HER2 Scoring of Breast Cancer Images Using Pyramid Sampling

Introduction: Breast cancer is the most common cancer among women worldwide and is a significant cause of cancer-related deaths. The accurate assessment of human epidermal growth factor receptor 2 (HER2) status is essential for treatment guidance. HER2 is a protein that plays a key role in the growth of cancer cells, with its expression levels providing valuable predictive and prognostic information. Traditionally, HER2 status is evaluated through manual inspection of immunohistochemically (IHC) stained images by pathologists. However, the manual evaluation of these slides is fraught with challenges, such as poor reproducibility and concordance among pathologists, which can compromise diagnostic accuracy. These issues are exacerbated in resource-limited settings where access to expert pathologists may be restricted, making HER2 assessment more challenging.
Given these complications, there is a growing need for automated tools to assist in HER2 status evaluation. Such tools aim to enhance the consistency of HER2 scoring. While there have been significant advances in computational methodologies for automated HER2 scoring, many current approaches do not adequately address the heterogeneity and complexity of tissue expressions typically seen in clinical settings. Most rely on small, selected image patches, which may not represent the broader context of the tissue involved, leading to potential overestimations of their effectiveness. Here, we introduce a deep learning (DL)-based system utilizing pyramid sampling to overcome these limitations by providing a comprehensive analysis of HER2 expression across different spatial scales, thus enhancing the diagnostic accuracy and efficiency of HER2 status assessment in breast cancer tissue images.

Materials and

Methods: We introduced a pyramid sampling strategy to address the heterogeneity of HER2 expression in breast cancer tissue. This method involves extracting and analyzing tissue patches at multiple resolution levels to capture a comprehensive range of morphological features. Specifically, we extract forty 512x512 pixel image patches from the original high-resolution image, ten patches from a 2x-downsampled image, and a single patch representing the resized entire tissue core, as illustrated in Fig. 1a. These patches are combined along the channel dimension to form a Pyramid-Sampling Set (PSS), serving as the input for our DL model.
The inference protocol for HER2 scoring is designed to ensure high reliability and precision. After the model generates a series of predictions for each tissue core, a subset of these predictions, known as the k-Confident Selection Set (KCS), is selected based on their high confidence levels. The final HER2 score is determined by the highest HER2 score within this KCS, reflecting the most probable HER2 status as indicated by the model’s most confident predictions.
To evaluate the statistical robustness of our approach, we conducted extensive testing with various configurations of the number of PSSs (N) and the confidence thresholds (k). These analyses, repeated 10,000 times for different settings, helped us identify the optimal operational conditions that balance accuracy with computational efficiency, ensuring the model’s practical applicability in clinical settings.

Results, Conclusions, and Discussions: We trained our model using a dataset of 1462 core images from 823 patients, validated it on 162 cores from 149 patients, and blindly tested it on 523 previously unseen core images from 300 patients. To ensure the integrity of our dataset, five board-certified pathologists independently scored the cores, using a voting protocol to manage discrepancies and ensure consistent labeling reflective of real-world scenarios. The model was validated through both qualitative and quantitative analyses, focusing on its capacity to handle HER2 expression heterogeneity. During the testing phase, each core image was converted into multiple independent PSSs, and the model predicted HER2 status by selecting the highest confidence scores. This approach not only addressed variability in tissue expression but also improved the reliability of HER2 scoring by focusing on the most confident predictions.
For each sample, we tested varying values of N and k to validate the consistency of our model. We observed a notable peak in classification accuracy when N was set to 20, reaching a maximum accuracy of 87.76%, with the median accuracy and minimum accuracy at 84.70% and 82.52%, respectively - see Fig. 1b. Beyond N=20, additional PSSs provided diminishing returns. Confusion matrices corresponding to the minimum, median (Fig. 1c), and maximum accuracy benchmarks provide a quantitative view of the system's performance.
Our method promises advancement in the field by utilizing a multi-scale feature analysis approach, a first in the automated scoring of HER2 in IHC-stained tissue images. The pyramid sampling strategy not only captures detailed cellular and broader tissue features but also ensures a comprehensive analysis that mitigates the risk of misclassification due to HER2 heterogeneity. Furthermore, the study highlights the practical implications of this technology in improving the consistency and efficiency of pathological assessments. By standardizing the scoring process, our system reduces the subjectivity associated with manual evaluations and significantly decreases diagnostic turnaround times, enhancing patient care by allowing faster and more reliable diagnostic decisions. Overall, this research moves us closer to widely accessible diagnostics, advancing the field of personalized medicine and improving oncological outcomes.

Acknowledgements (Optional):