Investigating Autoregressive Models and Period of Observation Length for Challenging Behavior Prediction in Autism Spectrum Disorder

Introduction: Autism spectrum disorder (ASD) affects 1 in 36 children according to the latest CDC estimate [1], and often co-occurs with other conditions [2], including gastrointestinal (GI) and sleep disorders. About a quarter of these children have profound ASD, which is characterized by being minimally verbal, requiring additional round-the-clock care and an IQ < 50. Those with profound ASD may not be able to communicate with their caretakers, and are more likely to exhibit challenging behaviors that are a danger to themselves and others, such as self-injurious behavior (SIB) and aggression. Predicting when these behaviors occur offers the opportunity for caretakers to prepare and prevent injury. We have shown in previous work that these behaviors can be predicted with at least modest accuracy for the vast majority of individuals, and a subgroup of 15-20% of these individuals has an average balanced accuracy exceeding 80%, which makes this clinically relevant for this subgroup [3]. Previously, we predicted behavior just using GI and sleep data from the previous day, in addition to some environmental factors, but as the previous day’s behavior is highly autocorrelated, this work incorporates prior behaviors into the model. As behaviors tend to change over time, this work investigates the effect of the time span used for building the model on the prediction accuracy. The general paradigm that more data tends to result in better models is challenged for this work due to inherent drifts present in the behaviors and how they relate to GI and sleep variables over time.

Materials and

Methods: The data for this work was collected at The Center for Discovery (TCFD), a nonprofit residential facility specializing in ASD and other complex disorders. Staff at TCFD record behavior, sleep, and GI data each day. A time period of up to 18 months was selected for this previously existing, de-identified data for 46 individuals. Included in the data was sleep duration and interruptions each night, the time, count, and Bristol scales of bowel movements (BMs) each day, and behavior episodes. Data was feature engineered and aggregated by day. If all other data except BM data was available, BMs were imputed for up to 3 days as “No BM”. Any episode of SIB or aggression was considered a behavior that day if all behavior shifts were recorded. The data was z-score normalized and fed into a kernel adaptive linear neuron model, training on 85% of data and testing on 15%, over a grid search of kernel parameters and randomly initialized weights. This process was performed for data for all 18 months and again for 6 months. The model with the best balanced accuracy on the test set was selected, and repeated for 30 random splits of data. Models were trained on the same data, with and without taking prior behavior into account as a variable, and model performance was compared by individual, and across time spans. For each individual, the median model was selected as representative and statistics were averaged across individuals in the 6 month and 18 month datasets.

Results, Conclusions, and Discussions: When using autoregression with up to 18 months of data, the autoregressive model performed slightly better, with 15% of individuals with a median model performance >= 80% and an averaged median balanced accuracy of 70%. The model without autoregression, in comparison, had 67% averaged median balanced accuracy and 10% of the same individuals with a median model performance >= 80%. The performance of the autoregressive model also increased slightly compared to the non-autoregressive model when using 6 months of data, however both models showed much stronger performance, with over 30% of individuals with a median model performance >=80% and a median balanced accuracy averaging over 75% across individuals. These numbers are significantly higher than was reported in our previous work, both due to the focus on the most important input variables for the model and the more focused timespan of data collection. Furthermore, this is the first example of an autoregressive model being used to predict challenging behavior on a day-by-day basis. Additionally, while previous literature has suggested 18 months as the timespan for data collection [3,4], we have found that 6 months of data produces substantially better model performance, indicating that recent GI and sleep patterns may contribute more heavily to challenging behaviors in this shorter period. The number of data points to be considered sufficient data for the model in this shorter time period should be further explored.

Acknowledgements (Optional):

References:
1. Maenner, M.J.; Warren, Z.; Williams, A.R.; Amoakohene, E.; Bakian, A.V.; Bilder, D.A.; Durkin, M.S.; Fitzgerald, R.T.; Furnier, S.M.; Hughes, M.M.; et al. Prevalence and Characteristics of Autism Spectrum Disorder Among Children Aged 8 Years — Autism and Developmental Disabilities Monitoring Network, 11 Sites, United States, 2020. MMWR Surveill Summ 2023, 72, 1–14, doi:10.15585/mmwr.ss7202a1.
2. Tye, C.; Runicles, A.K.; Whitehouse, A.J.O.; Alvares, G.A. Characterizing the Interplay Between Autism Spectrum Disorder and Comorbid Medical Conditions: An Integrative Review. Front Psychiatry 2018, 9, 751, doi:10.3389/fpsyt.2018.00751.
3. Ferina, J.; Kruger, M.; Kruger, U.; Ryan, D.; Anderson, C.; Foster, J.; Hamlin, T.; Hahn, J. Predicting Problematic Behavior in Autism Spectrum Disorder Using Medical History and Environmental Data. Journal of Personalized Medicine 2023, 13, 1513, doi:10.3390/jpm13101513.
4. Cohen, S.; Fulcher, B.D.; Rajaratnam, S.M.W.; Conduit, R.; Sullivan, J.P.; St Hilaire, M.A.; Phillips, A.J.K.; Loddenkemper, T.; Kothare, S.V.; McConnell, K.; et al. Sleep Patterns Predictive of Daytime Challenging Behavior in Individuals with Low-Functioning Autism. Autism Res 2AD, 11, 391–403, doi:10.1002/aur.1899.

Acknowledgements:
The authors would like to acknowledge Dr. John Erickson and Dr. James Hendler for providing computing cluster access, as well as the individuals, their families, and the staff at The Center for Discovery for providing the data.