Poster 80 - Gender-Specific Machine Learning Modeling for Sleep Stage Classification Using Wearable Device Data

Saturday, October 26, 2024

10:00 AM - 12:00 PM EST

Location: Exhibit Hall E, F & G

Presenting Author(s)

Jean Lee (she/her/hers)

Student
University of Maryland Baltimore County
Columbia, Maryland, United States

Co-Author(s)

Shaurya Bisht

Student
Thomas Jefferson High School for Science and Technology, Virginia, United States
GG

Grace X. Gao (she/her/hers)

Student
University of Maryland Baltimore County
Ellicott City, Maryland, United States

Introduction: Sleep is essential for physical and mental health, comprising more than a third of daily life [1, 2]. Sleep disorders can significantly impact sleep quality and often remain underdiagnosed; for example, approximately 80% of patients with sleep apnea have not been clinically diagnosed [3]. Polysomnography (PSG) is currently the gold standard for diagnosing sleep impairments by measuring signals including brain activity, eye movements, and heart rhythms. However, PSG measurements require human experts to assign sleep stages manually, and interscorer variability can lead to unreliable results [4]. Machine learning models have been designed to predict sleep stages instead of human experts, and could reduce the resources required to diagnose disorders. Past studies have shown machine learning models can achieve high accuracy on sleep stage classification compared to human labeling [5]. Sleep stage patterns and REM stage lengths vary based on gender, with males having shorter sleep latency and longer NREM occurrences [6]. Therefore, gender-specific machine learning models could improve performance and accuracy, as they are trained with a more specific dataset. Additionally, consumer-grade wearable devices measure signals similar to PSG using accelerometer and photoplethysmogram (PPG) data. Wearable devices are less invasive than PSG, which requires many sensors. A key advantage of wearable devices is the ability to conduct longitudinal studies, crucial to diagnosing disorders that present infrequently or tracking symptom progression. In this study, we trained gender-specific machine learning models with data from wearable devices to classify sleep stages, shedding light on the effect of gender on sleep prediction models.

Materials and

Methods: The Dataset for Real-time sleep stage EstimAtion using Multisensor wearable Technology (DREAMT) was used to train models for wake/sleep, wake/REM/NREM, and wake/N1/N2/N3/REM classification [7]. 100 subjects, 23 with severe obstructive sleep apnea, underwent PSG for one night while wearing the Empatica E4 wearable device. The dataset includes blood volume pulse (BVP), accelerometer, electrodermal activity (EDA), temperature data from the wearable device, and ground truth sleep stage labels every 30 seconds from PSG human scoring.

During preprocessing, BVP data was filtered with a Chebyshev type 2 band pass filter, and the EDA data was detrended and filtered with a Butterworth low pass filter [8]. The HR data was normalized and outlier values of more than 5 standard deviations were removed from the IBI data. Heart rate variability (HRV) and skin conductance response (SCR) features were both derived from the BVP and EDA signals. Statistical features such as mean, median, and standard deviation were extracted from all signals, shown in Figure 1.

Gender-specific models were trained on a single gender dataset and compared to gender-blind models to determine performance differences. The Synthetic Minority Oversampling Technique (SMOTE) was used to generate synthetic data points (kNN set to 5 neighbors) for the minority classes, as there was a significant class imbalance, as shown in Figure 2. Logistic regression, Least Absolute Shrinkage and Selection Operator (LASSO) regression, and Random Forest models were trained and evaluated with leave-one-subject-out cross-validation. LASSO regression was used to assess the interoperability of models.

Results, Conclusions, and Discussions: Random forest models achieved the greatest performance for all classification tasks and evaluations. In sleep/wake classification, the random forest gender-blind model achieved an AUC of 0.95 when tested on both the male and female datasets. Performance declined slightly when the number of classes increased, as seen in Figure 3. In Figure 5, wake/N1/N2/N3/REM classification received scores of 0.90 and 0.91 when tested on the male and female datasets respectively.

Performance always increased when gender-specific models were trained and tested on the same gender. For sleep/wake classification, both random forest gender-specific models achieved the greatest results with an AUC of 0.98. Gender-specific models trained on female data generally performed better across most evaluations than the male-specific models.
LASSO regression identified HR, circadian, and accelerometry metrics as key features for classification, with total power being especially crucial for females.

In wake/REM/NREM classification, we observed a greater drop in performance for the logistic regression model as shown in Figure 4. REM classification was the most challenging, whereas wake was the least difficult. Important features included IBI (especially for females), circadian, and accelerometry metrics.

In wake/N1/N2/N3/REM classification, the models had significantly lower accuracy in predicting N1 and N3 stages. This could be due to the N1 and N3 stages being minority classes in our dataset. However, this was partially expected since human labelers also have the highest disagreements on N1 and N3 stages due to low occurrences and similar sleep patterns [2].
LASSO regression indicated that essential features for predicting each sleep stage were as follows: HR and circadian features for N1, IBI and circadian features for N2, circadian features for N3, and HR and circadian features for REM.

In this study, gender-specific models performed better than gender-blind models. However, the range of performance benefits varied across models and classification problems and should be considered in future research. To our knowledge, our study is the first to use wearable data from a dataset focusing on sleep disorders to train gender-specific sleep stage classification models.

Acknowledgements (Optional): We would like to thank Dr. Lujie Karen Chen and Mr. Sean Kong for providing the knowledge necessary to complete this project, as well as for reviewing our work and assisting throughout the writing process.

We extend our thanks to the creators of the DREAMT dataset: Ke Wang, Jiamu Yang, Ayush Shetty, and Jessilyn Dunn. This abstract would not be possible without you.

We also thank Will Wang, Jerry Yang, Angelica Pan, and Hayoung Jeong, who created an insightful notebook that was fundamental for our preprocessing and feature engineering.

We would also like to thank the professionals whom we presented to and gained feedback from including but not limited to: Dr. Neeraj Badjata and Dr. Peter Hu.

References (Optional): [1] Uccella S, Cordani R, Salfi F, Gorgoni M, Scarpelli S, Gemignani A, Geoffroy PA, De Gennaro L, Palagini L, Ferrara M, Nobili L. Sleep deprivation and insomnia in adolescence: implications for mental health. Brain sciences. 2023 Mar 28;13(4):569. https://doi.org/10.3390/brainsci13040569.
[2] Chokroverty S. Overview of sleep & sleep disorders. Indian Journal of Medical Research. 2010 Feb 1;131(2):126-40. http://dx.doi.org/10.1016/S0030-6665(05)70123-7
[3] Young T, Evans L, Finn L, Palta M. Estimation of the clinically diagnosed proportion of sleep apnea syndrome in middle-aged men and women. Sleep. 1997 Sep 1;20(9):705-6. https://doi.org/10.1093/sleep/20.9.705.
[4] Rosenberg RS, Van Hout S. The American Academy of Sleep Medicine inter-scorer reliability program: sleep stage scoring. Journal of clinical sleep medicine. 2013 Jan 15;9(1):81-7. https://jcsm.aasm.org/doi/10.5664/jcsm.2350.
[5] Walch O, Huang Y, Forger D, Goldstein C. Sleep stage prediction with raw acceleration and photoplethysmography heart rate data derived from a consumer wearable device. Sleep. 2019 Dec;42(12):zsz180. https://doi.org/10.1093/sleep/zsz180.
[6] Mallampalli MP, Carter CL. Exploring sex and gender differences in sleep health: a Society for Women's Health Research Report. Journal of women's health. 2014 Jul 1;23(7):553-62. https://doi.org/10.1089/jwh.2014.4816.
[7] Wang K, Yang J, Shetty A, Dunn J. DREAMT: Dataset for Real-time sleep stage EstimAtion using Multisensor wearable Technology. https://doi.org/10.13026/62AN-CB28.
[8] Wang W, Yang J, Pan A, Jeong H. DREAMT Case Study [Internet]. Mountain View, CA: Google; 2024 [cited 2024 Aug 15]. Available from: https://colab.research.google.com/drive/10ou6z38SC-4gKIW5p_5fQZlLhOKdeG0N?usp=sharing.