Identification of Fiducial Points on 3D Torso Images Using Contrastive Learning

Introduction: The utilization of three-dimensional (3D) imaging is expanding for quantifying breast morphology, particularly for assessing breast symmetry and for longitudinal monitoring of changes in breast appearance during reconstructive surgery. Breast morphometry from 3D photographs has been reported, however, several studies utilize manual identification of anatomical landmarks (fiducials), which is time-consuming and subject to operator bias. Using machine learning (ML) algorithms to automate the identification of fiducials would mitigate operator bias and yield reliable, objective measurements. Here, we propose an ML framework to automate the identification of the sternal notch (SN) and umbilicus (U) in 3D photographs.

Materials and

Methods: An ML framework was developed to detect individual fiducial points using supervised contrastive learning. The framework includes a feature extractor utilizing a convolutional neural network backbone and a multi-layer perceptron to generate and condense a feature map for each image. Specifically, we utilized a feature bank that is updated regularly during training to optimize the representation of fiducial points and filter out background (clutter) points. This framework ensures that features of the same fiducial point are similar to each other but distinct from features of other fiducial points and background clutter. Inference is performed by computing a score map for each fiducial point, where the highest score indicates the predicted point location. We used 3D photographs taken before and after breast reconstruction from 273 patients. The dataset was randomly sampled, with 191 images used for training, 27 images for validation, and 55 images for testing. The dataset images were divided among two graduate students trained by surgeons to select fiducials. Each student manually identified and recorded the fiducial locations on their assigned set of images using custom software. We define as ground truth (GT) these locations used as reference during training, validation, and testing. The framework accuracy was evaluated using the percentage of correct keypoints (PCK), which considers a fiducial to be correct if its L2 distance from the GT is within a chosen radius determined from previously reported anthropometric data (20mm for SN[1],[2] and 12.5mm for U[3]). We report the average distance between fiducial predictions and GT for evaluation.

Results, Conclusions, and Discussions: The framework detects fiducial points in 75.76% of cases (SN in 80.51% of cases and U in 71.01% of cases). Table 1 provides a comparison on the performance of fiducial identification approaches in torso images against the proposed approach. Kenig et al. [4] proposed a deep learning method achieving 87% accuracy for SN identification in 2D breast surgery patients’ frontal photographs, leveraging properties like color texture and grid structure that are not available in our 3D point clouds dataset. Kawale et al.'s [5] approach utilized surface curvature measurements on 3D meshes, reporting 36.36% and 63.64% accuracy for SN and U respectively, with higher average predicted SN to GT distance compared to ours which indicates that our framework provides better SN predictions. While their method achieves closer U predictions to GT, it requires manual registration of torso meshes to upright frontal position, unlike our approach.
As shown in Table 2, the framework provides higher accuracy for patients in the healthy and overweight BMI groups compared to the obese group.
Figure 1 presents a visualization of the score maps for each fiducial point in 3D torso images of patients within different BMI groups. The highest scores (red-yellow shades) correspond to points surrounding the ground truth fiducial point.
This framework was successful in detecting fiducial points in 3D surface scans of breast surgery patients. Mean detection time was 0.85 seconds. The proposed framework can be extended to identify additional fiducial points on the torso. Moreover, it can be integrated in various medical applications, such as the registration of 3D torso images from different clinical visits and the evaluation of breast symmetry in plastic surgery.

Acknowledgements (Optional): This study was supported by NIH grant R01 CA203984 and R41 CA272122.

References:

[1] P. P. Mukhopadhyay, “Determination of Sex from Adult Sternum by Discriminant Function Analysis on Autopsy Sample of Indian Bengali Population: A New Approach,” J Indian Acad Forensic Med, vol. 32, no. 4, pp. 321–324, 2010.
[2] R. Baraw, J. Barwa, M. Pradhan, and S. K. Khanna, “Width of sternum at suprasternal notch and at the level of fourth rib for determination of sex in the population of Delhi: A new & reliable tool for sex prediction,” Journal of Indian Academy of Forensic Medicine, vol. 39, no. 1, pp. 83–87, 2017, doi: 10.5958/0974-0848.2017.00018.5.
[3] B. R. Baack, G. Anson, J. M. Nachbar, and D. J. White, “Umbilicoplasty: the construction of a new umbilicus and correction of umbilical stenosis without external scars,” Plast Reconstr Surg, vol. 97, no. 1, pp. 227–232, Jan. 1996, doi: 10.1097/00006534-199601000-00038.
[4] N. Kenig, J. Monton Echeverria, and L. De la Ossa, “Identification of Key Breast Features Using a Neural Network: Applications of Machine Learning in the Clinical Setting of Plastic Surgery,” Plast Reconstr Surg, vol. 153, no. 2, pp. 273E-280E, Feb. 2024, doi: 10.1097/PRS.0000000000010603.
[5] M. M. Kawale et al., “Automated Identification of Fiducial Points on 3D Torso Images,” Biomed Eng Comput Biol, vol. 5, p. BECB.S11800, Jan. 2013, doi: 10.4137/BECB.S11800.