Skip to main content
2024 BMES Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Back

Neural Engineering

Neural Engineering - Poster Session A

Poster J19 - An Intuitive and High Accuracy Hybrid Brain Machine Interface for Telepresence Robot Control

Thursday, October 24, 2024
10:00 AM – 11:00 AM EST
Location: Exhibit Hall E, F & G

Introduction:  Brain-machine interfaces (BMIs) can significantly improve the quality of life for people living with severe motor impairments by enabling control of assistive devices, such as telepresence robots, through neural signals. Such devices promote the independence and self-determination of these individuals. In BMIs, two of the most commonly used non-invasive methods are surface electromyography (EMG) and surface electroencephalography (EEG). Surface EMG records neuromuscular activity from the skin, while surface EEG captures cortical brain activity from the scalp.

However, either EMG or EEG alone has limitations. Prolonged EMG use can lead to muscle fatigue and signal deterioration, while continuous cognitive control with EEG often results in mental fatigue and often requires weeks of training to achieve satisfactory accuracy. These factors make traditional EMG- or EEG-based BMIs challenging to learn and use, especially for controlling complex systems like telepresence robots.

This study introduces a novel hybrid BMI that employs both EEG steady-state visually evoked potentials (SSVEP) and EMG signals from cheek and neck muscles to control the locomotion as well as camera position of a telepresence robot. The system is able to decode user intention from EMG activity with over 95% accuracy and SSVEP with up to 100% accuracy. Furthermore, the system has potential to mitigate fatigue by allowing users to alternate between EEG and EMG, thereby reducing the continuous use of either. These results demonstrate the potential for hybrid BMIs to enhance the usability and effectiveness of assistive technologies, thereby improving the quality of life for individuals with severe motor impairments.

Materials and

Methods:  Users have access to two functions of the telepresence robot: locomotion and camera position, each having 2 degrees of freedom. Users switch the control paradigm by using an 8-channel EEG to produce SSVEP though focusing on one of the two SSVEP stimuli, each flashing at 9.75 Hz and 14.25 Hz. Control of the robot locomotion/camera view angle are executed via a 2-channel EMG (Fig. 1).

Two experiments, non-simultaneous and simultaneous EMG & EEG, are designed to collect data for training and testing the decoding algorithms (Fig. 2). The non-simultaneous experiment provides independent EMG and EEG data, whereas the simultaneous experiment provides intermixed EMG and EEG data and is more similar to the intended use case of the hybrid BMI.

Filters are applied to preprocess SSVEP and EMG signals. For SSVEP signals, a 2 Hz high pass filter and a 60 Hz notch filter are implemented using infinite impulse response (IRR) filters to preserve harmonics. For EMG signals, a 50-150 Hz bandpass exponential moving average (EMA) filter is applied. The EEG sensors sample at 250 Hz, while the EMG sensors sample at 500 Hz.

To decode the EMG signal, two separate logistic regression models are used to classify whether or not the cheek or neck muscles are contracting separately by comparing the signal with a threshold. The EMG classifiers are trained on rolling windows of 200 ms with an 80 ms overlap. A canonical correlation analysis (CCA) model is used to decode SSVEP stimuli from a 3-second window of EEG signals.

Results, Conclusions, and Discussions: A time-frequency representation (TFR) (Fig. 3a) of the SSVEP signal for Subject 01 reveals clear differences in the distribution of frequency power for trials where the subject focused on the 9.75 vs. 14.25 Hz signal. These results are consistent with existing literature on the neuroscientific principles behind SSVEP. Analysis of the EMG classifier (Fig. 3b) compares the timing of the classifier’s prediction of a muscle contraction compared to the true label for a sample 200 ms window of EMG data. It can be observed that the prediction aligns well with the actual EMG signal bundle as well as the true label.

Our experimental data from 3 subjects (Table 1) demonstrates that in the non-simultaneous experiments the system is capable of achieving up to 87% EMG decoding accuracy and up to 100% SSVEP decoding accuracy in an experienced user. Notably, it also achieves over 90% EMG decoding accuracy and over 80% SSVEP decoding accuracy in first-time users, demonstrating its usability. In the simultaneous experiment, the system is capable of achieving over 76% accuracy in EMG decoding and 98% in SSVEP decoding for the experienced user, validating its performance for its intended use case. These results together suggest that the system can properly decode user intention from EMG and SSVEP signals with minimal training, making it accessible and user-friendly.

However, the significance of these results is currently limited by the small sample size and the fact that we have only tested the system on individuals without motor disability. To address these limitations, we will extend our experiments to include a larger and more diverse group of subjects, both with and without motor disability, to enhance the robustness and generalizability of our data.

Additionally, we are in the process of using the NASA Task Load Index (TLX) to analyze the extent to which the hybrid approach reduces user fatigue compared to using only EEG or EMG.

Overall, our findings indicate that the hybrid BMI system holds significant promise for intuitive and high-accuracy control, and further research is needed to validate the efficiency across diverse populations and to understand its impact on user fatigue.

Acknowledgements (Optional):  We would like to extend our heartfelt gratitude to Dr. Sarma and Dr. Buccafusca for their guidance and support throughout this project. We are also sincerely grateful for the generous financial support from the JHU HOUR Office and the JHU WSE Alumni Association.

In addition, we are sincerely grateful for the help and support from Jixin (Audrey) Zheng as a mentor and a Hopkins alumnus.