Exploratory Comparison of Human Sensorimotor Event-Related Desynchronization Responses during Observations and Imitations of Actions Performed by Human and Robot

Introduction: Action observation (AO) therapy represents a promising method for rehabilitation that effectively enhances motor function in upper extremities and gait for individuals recuperating from neurological conditions like stroke. This therapy capitalizes on the mirror neuron system (MNS), which is activated during both the observation and execution of motor acts. Observing others' actions triggers similar motor activation as executing those actions oneself. While social robots have been increasingly integrated into rehabilitation to boost patient motivation and engagement and to alleviate therapist workload, their potential role in AO therapy remains unexplored. Researchers have questioned whether artificial agents like robots can induce comparable responses to humans. Some studies examined MNS via observations of electroencephalography (EEG) data in healthy individuals observing human and robot actions [1,2]. They used event-related desynchronization (ERD) (i.e., the decrease in power of brain activity relative to a baseline) of mu (8-13 Hz) and beta (13-30 Hz) oscillations at sensorimotor areas to indicate the MNS activation and found similar MNS reactivity to both types of actors, suggesting the potential of robots-assisted AO therapy. However, limited research on neural responses during the observation and imitation of humanoid robots' actions hinders definitive conclusions about the efficacy of robot-assisted MNS-based therapy. To address this gap and lay groundwork for future research, a case study was conducted with three healthy participants performing AO and action execution (AE) tasks with humanoid robots and human actors. Analysis of ERDs may shed light on the humanoid robot's influence on MNS and validate EEG-based sensorimotor activity identification.

Materials and

Methods: The stimuli comprised 5000-ms videos depicting various unimanual upper limb actions from an allocentric viewpoint. Each video featured one action performed by a custom-designed humanoid social robot named Flo [3] and two human actors (one male and one female), utilizing their left and right arms. During AO experiment, participants observed a randomized sequence of these videos, each video preceded by a 1000-ms fixation cross serving as baseline EEG. Two experimental conditions were “human” and “robot” actions and “control” conditions involved landscape videos. In AE experiment, participants observed the same “human” and “robot” videos in random order, then imitated the action within 4000 ms after fixating on a cross for 1000 ms. There were no “control” trials in AE experiment. EEG data were recorded during imitation period. Offline, EEG data underwent filtering (1–40 Hz) and artifact removal by visually eliminating channels containing artifacts. The data were then re-referenced to the average, with interpolation of removed channels, and cleaned using Independent Component Analysis (ICA). Data were segmented into 3500-ms epochs (beginning at action onset for AO and imitation period for AE), and epochs with amplitude exceeding 100 µV were rejected. Time-frequency analyses assessed sensorimotor mu and beta ERDs relative to baseline power, averaged across human actor types, observed arms, and motor-related channels (C3 and C4), separately for each frequency band, condition, and experiment. To capture the temporal dynamics, epochs were divided into multiple 250 ms windows, and ERDs were computed for each window.

Results, Conclusions, and Discussions: Results and

Discussion: Grand average of time-frequency decomposition of power across all participants and conditions found significant ERD within mu (8-13.5 Hz) and beta (17-24 Hz) bands for AO and mu (9-13 Hz) and beta (19-24 Hz) bands for AE. Subsequent analyses of ERDs at these identified frequency bands in Fig. 1 showed consistent patterns of ERDs for mu and beta bands for the experimental conditions, noticeably starting from 500 ms (except for “human” at beta band during AE), with stronger mu ERDs in AE than in AO. These results of mu ERDs are consistent with findings in Duann and Chiou’s study [4]. During AO, ERDs for 2 experimental conditions were not significantly different, which is congruent with previous studies [1, 2]. However, there were no significant differences in mu and beta ERDs between the two experimental conditions and “control” videos, which might suggest the neuromodulation of areas in visuomotor system instead of the true MNS activation during AO. During AE, significantly stronger ERDs for “robot” compared to “human” condition were seen at both mu and beta bands, from 1000 ms to 1750 ms.

Conclusion: To our knowledge, this is the first study investigating the temporal dynamics of robot-assisted AO and AE tasks. Results suggest the consistency of ERD patterns throughout the task for most “human” and “robot” conditions. The results of comparison of ERD responses during observations of actions performed by human and robot actors align with the literature [1,2], supporting the finding that the human MNS is not exclusively selective for human agents. The stronger ERDs during AE for the "robot" condition may indicate the neuromodulation effect during physical human-robot interaction, specifically participants' imitation of actions by the robot in this study. Future research will evaluate EEG data from individuals with neurological conditions such as stroke patients to confirm the potential effects of using social robots in stroke neurorehabilitation and promote the efficacy of robot-assisted MNS-based therapy.

Acknowledgements (Optional): Acknowledgments: This work was supported by the Department of Physical Medicine and Rehabilitation at the University of Pennsylvania and by the Eunice Kennedy Shriver National Institute of Child Health & Human Development of the National Institutes of Health (NIH) under Award Number F31HD102165 and the Vingroup Scholarship Program for Master’s and Ph.D. Degrees Overseas Study. The content does not necessarily represent the views of the NIH.

References: [1] Urgen, Burcu A., et al. "EEG theta and Mu oscillations during perception of human and robot actions." Frontiers in neurorobotics 7 (2013): 19. [2] Oberman, Lindsay M., et al. "EEG evidence for mirror neuron activity during the observation of human and robot actions: Toward an analysis of the human qualities of interactive robots." Neurocomputing 70.13-15 (2007): 2194-2203. [3] Sobrepera, Michael J., et al. "The design of Lil’Flo, a socially assistive robot for upper extremity motor assessment and rehabilitation in the community via telepresence." Journal of Rehabilitation and Assistive Technologies Engineering 8 (2021): 20556683211001805. [4] Duann, Jeng-Ren, and Jin-Chern Chiou. "A comparison of independent event-related desynchronization responses in motor-related brain areas to movement execution, movement imagery, and movement observation." PLoS One 11.9 (2016): e0162546.