Explainable AI for Thyroid Cancer Detection using Smartphone-captured Cytopathology Images and Multi-Instance Learning

Introduction: Digital pathology is transforming the traditional pathology. It allows remote clinical case diagnosis and application of advanced image analysis algorithms, including AI-based algorithms. These algorithms can potentially support doctors in diagnosis and treatment and improve clinical care and outcomes. However, digitizing slides requires expensive scanners costing up to $250,000, a great cost barrier for many clinical centers in LMICs. With lack of equipment and shortage of pathology expertise, people living in LMICs, especially outside big cities, are not benefiting from the advancements in the field. Our group previously proposed a low-cost smartphone-based digitizing approach for capturing cytopathology slide images followed by AI-powered analysis for thyroid cancer detection with accuracy of 96% on smartphone images collected at Duke Hospital, USA. In this work, we tested the approach in two LMICs countries, Tanzania and Vietnam, and made two main contributions including: (1) propose AI models based on EfficientNet and multi-instance learning that can accurately predict malignancy from smartphone-captured cytopathology images from Tanzania and Vietnam with accuracy of 90% and 95% respectively, (2) the models’ predictions are presented with heatmaps showing regions that are important to the models’ predictions, providing explainability to our models (Figure 1). Explainability in AI model’s predictions is important for successful deployment of AI models in medicine. We believe that our smartphone and AI-based approach have the potential to be integrated in clinical workflow in LMICs to improve thyroid cancer clinical care and outcomes in these countries. The approach can also be applied for other cancer types and diseases.

Materials and

Methods: Data were collected from the Kilimanjaro Christian Medical Center (KCMC) in Moshi, Tanzania and the National ENT Hospital in Hanoi, Vietnam. The Tanzania site used Pap stain and the Vietnam site used alcohol-fixed stain. In both sites, cytopathology slide images were captured by Redmi Note 10S mobile phone cameras (Xiaomi Inc) attached to optical microscopes. In total, 1,119 images from 100 patients were collected from Tanzania site and 2,048 smartphone images from 132 patients were collected from Vietnam site (Figure 2). The images were resized and crop to 224 x 224 px and divided into a training set and a testing set in patient-preserved manner. All patients have cytopathologist diagnosis and surgical diagnosis. We used surgical diagnosis, malignant or benign, of each patient to assign label for all images belong to the patient (i.e., weakly supervision). Due to different staining methods and optical equipment, image characteristics from the two sites were noticeably different from each other (Figure 3) and from Duke Hospital’s smartphone-captured slide images. We therefore independently trained and tested two AI models based on Efficient-b0 architecture with ImageNet pretrain weights, one on Tanzania images and the other on Vietnam images. The models were trained on a local machine for 100 epochs using PyTorch framework. Data augmentation including random crop and flip were using during training. CrossEntropyLoss and Adam were used as loss function and optimizer respectively.

Results, Conclusions, and Discussions: For Tanzania data, the AI achieved 90% accuracy and 91% weighted average F1-score (Figure 4A) with 100% sensitivity and 89% specificity of during testing. For Vietnam data, the AI achieved 95% accuracy and 95% weighted average F1-score (Figure 4B) with 97% sensitivity and 89% specificity during testing. We attempted to explain the models’ predictions using an Occlusion-based approach and the Captum library. The results showed that for most of the images, the models correctly looked at cell regions when making predictions (Figure 5). Many AI models reported so far lack explainability, a problem also known as “black box” issue. Without explanation, user tend to less trust on AIs’ predictions, which in turn could hinder AI deployment in clinical workflow. Our models not only exhibit high accuracy on data collected from two LMICs, Tanzania and Vietnam, but also their predictions can be explained via heat maps highlighting regions that are important to the models’ predictions. Model’s explainability is important, especially for applications in medicine. The use of smartphone attached to microscope for capturing slide images significantly reduced the digitizing cost. This approach is promising and have potential for application and improving thyroid cancer diagnosis in LMICs.

In conclusion, we report two AI models based on EfficientNet and multi-instance learning for thyroid cancer detection using smartphone-captured cytopathology images collected from two LMICs, Tanzania and Vietnam. The models exhibited 90% and 95% accuracies on Tanzania test set and Vietnam test set, respectively. The models’ predictions can be explained via heatmaps highlighting regions at which the models are looking. Our smartphone-based imaging + AI analysis approach can potentially be a solution to make AI-powered digital pathology more accessible to laboratories in LMICs.

Acknowledgements (Optional): The authors graciously acknowledge funding from the National Institutes of Health (1R21CA268428-01 NC)