Skip to main content
2024 BMES Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.

Biomechanics

Biomechanics - Poster Session A

Poster D9 - Assessing the effect of increased training data on automatic segmentation performance for small upper limb muscles

Thursday, October 24, 2024
10:00 AM - 11:00 AM EST
Location: Exhibit Hall E, F & G

Introduction:  Medical imaging is a valuable tool in musculoskeletal research capable of producing large quantities of data to answer questions relating to patient health, muscle function, or physiological structure [1]. Particularly, muscle volume segmentation can have applications in areas ranging from surgical planning, to sports performance optimization, to monitoring rehabilitation progress [2,3]. Traditionally, experts manually identify structures of interest [1,4,5], but manual segmentation of individual muscles from numerous slices of computed tomography, magnetic resonance or ultrasound images is time consuming and tedious, which has limited its implementation [6]. Machine learning has emerged as a novel technique for automatic segmentation of medical images [7,8]. Despite the success of previous deep convolutional neural networks (DCNNs) in automatic muscle segmentation, segmentation accuracy is known to be sensitive to muscle size, with small muscles achieving lower performance metrics [7]. This study aimed to evaluate segmentation performance results, with a focus on smaller muscles, through the addition of two new datasets to a model previously trained on one dataset. Cross validation was implemented to evaluate model performance on external datasets, and to gauge model generalizability.

Materials and

Methods:  Upper limb muscle volumes that were manually segmented in three previous studies were used for DCNN training and testing; dataset A: 10 subjects, dataset B: 10 subjects, dataset C: 19 subjects [6,9,10]. Proximal muscles (those crossing the shoulder or elbow) were acquired in a single scan with a body coil; distal muscles (those in the forearm crossing the wrist or elbow) were acquired with an arm coil. All muscles in datasets A and B were captured via a single scan. In dataset C, some distal muscles required two overlapping scans due to scanner size limitations.

Dataset curation involved conversion of manual segmentations to binary and multiclass masks in NeuroImaging Informatics Technology Initiative (NIfTI) format. Initial training results from the most complete dataset (dataset A), were used, alongside muscle volumes, to identify 5 small muscles (volume < 1.5% of total muscle volume in the upper limb) with poor segmentation performance (validation dice < 0.8) for analysis (Fig. 1).

The nnUNet python library [11] was used to train DCNN models on distal and proximal muscles separately. Each model was trained for 1,000 epochs with 5-fold cross validation. To evaluate how adding training data affected model performance, models were trained using segmentations from combinations of the three datasets. In addition, the model trained on dataset A was tested on dataset B to assess robustness and generalizability. Dice scores and muscle volume errors across validation or test data were used as metrics to assess model performance and determine segmentation prediction accuracy.

Results, Conclusions, and Discussions: Adding high quality data, of continuous 3D muscle volumes, was generally found to improve small muscle segmentation performance. The DCNN trained and internally validated on images from datasets A and B performed better than the DCNN trained on images from dataset A alone (cf. Fig. 2: red line is above blue line for all small muscles). Similarly, the best model for segmenting proximal muscles contained all three datasets (cf. Fig. 2: green line above red and blue lines for small proximal muscles). The same improvement in performance metrics was not seen when adding dataset C images for training on small distal muscles. The imaging artifacts associated with data collection across two scans in dataset C likely impacted the model’s learning ability (cf. Fig. 2: green line below red and blue lines for small distal muscles). Despite the improvement in validation dice scores with the addition of more data, the small muscle dice scores still lag behind larger muscle dice scores. The improvement in segmentation performance, however, was more prominent for smaller muscles. These results indicate that inclusion of more manual segmentation data, either from new data collection or segmentation of additional muscles in datasets B and C, would be valuable for continuing the improvement of small muscle segmentation performance.

Evaluation of the initial model trained on dataset A and tested on dataset B yielded promising results for model generalizability, with most small muscles achieving dice scores between 0.6 and 0.8 (Table 1). Teres major (TMAJ), however, yielded a dice score of 0.075 and volume error of 36.2%, indicating possible incompatibility or inconsistency in the segmentation of this muscle between the two datasets. Further external validation is needed to evaluate how the generalizability of trained models is affected by the inclusion of multiple training datasets (A, B, and C). Such experiments would test the hypothesis that additional training data improves model robustness, in addition to improving model performance.

Acknowledgements (Optional):  This work is supported by NIH R21AR080953 and Northwestern’s Summer Undergraduate Research Grant (SURG). This research was supported in part through the computational resources and staff contributions provided for the Quest high performance computing facility at Northwestern University which is jointly supported by the Office of the Provost, the Office for Research, and Northwestern University Information Technology.

References:
[1] H. E. Berg et al., Eur. J. Appl. Physiol. 99 (2007); [2] J. Pitoccchi et al., J. Shoulder and Elbow Surg. 30 (2020); [3] G. G. Handsfield et al., Scand. J. Med. Sci. Sports 27 (2017); [4] M. J. Tingart et al., Clin. Orthop. Relat. Res. 415 (2003); [5] E. A. Audenaert et al., Med. Biol. Eng. Comput. 47 (2009); [6] K. R. S. Holzbaur et al., J. Biomech. 40 (2007); [7] R. Ni et al., J. Med. Imaging. 6 (2019); [8] L. Riem et al., Radiol. Artif. Intell. 5 (2023); [9] K. R. Saul et al., PubMed. 31 (2015); [10] M. E. Vidt et al., J. Biomech. 45 (2012); [11] F. Isensee et al., arXiv:1809.10486 (2018);