Skip to main content
2024 BMES Annual Meeting Main banner
Icon Legend
This session is not in your schedule.
This session is in your schedule. Click again to remove it.
Back

Biomedical Imaging and Instrumentation

Biomedical Imaging and Instrumentation - Poster Session E

Poster F6 - Towards Wearable Forearm Ultrasound Based Gesture Recognition for Human-Robot Interfacing

Saturday, October 26, 2024
10:00 AM – 11:00 AM EST
Location: Exhibit Hall E, F & G

Introduction:  Our research focuses on developing a lightweight, cost-effective wearable device with non-invasive functionality that utilizes ultrasound probes to capture real-time forearm muscle images, enabling a robot to mimic precise and accurate hand gestures. The research focuses on the utilization of the WULPUS system (Wearable Ultra-Low-Power Ultrasound) as it is a cost-efficient and lightweight wearable device. In addition, the research utilized Verasonics Software for reliability of the WULPUS system.

Materials and

Methods:  Two ways of studying hand gestures were used in this research. Both processes include placing the ultrasound probe on the forearm placed in anatomical position and studying the muscles that are used when performing two hand gestures: open and closed motions of the hand. The first method of study was the Verasonics Software. It used a single element ultrasound probe. The second method of study used the WULPUS system which utilized an ultrasound transducer.
WULPUS consists of an Acquisition PCB designed for ultrasound measurement control and data management, complemented by a High-Voltage PCB for ultrasound transducer driving and signal multiplexing, thereby enhancing the devices modularity and functionality [1]. WULPUS not only has hardware components but also uses a Python library and Graphical User Interface (GUI) for logging and processing data then allowing the data to be visualized through graphical interfaces [2].
WULPUS allows for communication with robotic arms, enabling accurate replication of hand gestures such as performing open and closed movements. Post results show that the data acquired from Verasonics, established a baseline comparison of the muscle activity when performing different hand gestures. The baseline study used a single-element ultrasound probe that analyzed 500 frames per gesture. The WULPUS system provided a set of data called B-mode data for comparing the two types of hand gestures, open and closed. B-mode allowed for 2D imaging of the muscles in the forearm. This provided real-time imaging of the muscle in reference to the transducer.

Results, Conclusions, and Discussions: In the WULPUS B-Mode data images, the closed hand gesture demonstrated that there is a higher intensity of signal within the forearm muscles. This is due to being positioned closer to the ultrasound probe. The open hand gesture reveals more gaps and fewer waveforms compared to the closed hand gesture. This indicates the further away the muscle activation is from the ultrasound probe, the less activation is shown. Using a neural network program through Verasonics, it showed a 90% accuracy of hand gesture identification. This confirms the data acquired from the WULPUS system. Future developments, within training the neural network to classify additional hand gestures, will refine the robot’s accuracy and precision when mimicking the controller’s arm movements.

Acknowledgements (Optional):  Acknowledgements: This work was supported and funded through the NSF (Grant: EEC2150076), ETH and WPI. I would also like to acknowledge Shang Gao, Yichuan Tang, and Keerthana Cheelamanthula for their support on the project.