Abstract
Project Title: VR-BCI4PM: A Virtual Reality System Controlled by a Hybrid Brain-Computer Interface to Improve Powered Mobility in Individuals with Neuromotor Disorders Abstract: Individuals with spinal cord injuries (SCI), traumatic and acquired brain injuries (ABI), cerebral palsy (CP), multiple sclerosis (MS), amyotrophic lateral sclerosis (ALS), spinal cord injury (SCI) may have limited mobility and often require power wheelchairs for conducting an independent life. Powered mobility is a viable option, but requires strenuous training to guarantee safe driving conditions. Usually, wheelchair training is conducted with therapists at the hospital, with considerable costs for the national health system. Also, 10-40% of people cannot use a power wheelchair due to sensory, motor, and neurocognitive impairments. These people are considered unable to safely drive and are forced to use manual wheelchairs or ask for caregivers’ support. Driving skills and types of aid suitable for independent mobility are established based on the Powered Mobility Program (PMP). However, to date, no clinically validated tools support the user’s training to fulfill the PMP. Virtual reality (VR) constitutes a portable solution to perform safe training at home. To date, no existing VR simulators have been developed to assess users’ driving skills. Also, the existing VR simulators allow controlling movements with joysticks or hand trackers, but they are unusable for individuals with severe upper limb motor impairments. In this context, brain-computer interfaces (BCIs) represent a potential candidate as an innovative control interface. Although previous studies have explored the development of multimodal approaches integrating joysticks, electromyography (EMG), and eye-tracking, there are no training systems based on hybrid BCI solutions that decode information from electroencephalography (EEG) and EMG and that involve a wider audience. Herein, we will develop an innovative simulator based on a VR platform (the VR-BCI4PM) and we will longitudinally evaluate the improvement of wheelchair driving performance with 20 participants suffering from CP, ALS, MS, ABI, or SCI. A self-paced hybrid BCI will be developed for VR joystick-free control. Through 10 driving sessions with VR-BCI4PM at home (if the participant can use the joystick) or in the clinic (if the BCI controller is required), we will evaluate the effectiveness of the simulator by comparing the pre-and post-training results using the PMP protocol as ground-truth. Furthermore, a calibration system for VR-BCI4PM will be developed to compare the VR metrics to real-world conditions during on-road wheelchair tests. Lastly, usability, level of satisfaction, stress due to mental workload, and motion sickness will be assessed via behavioral questionnaires and physiological signals. Not only will VR-BCI4PM improve the wheelchair driving skills of people with severe motor impairment, but it will also establish objective measures to personalize the training, with the ultimate goal of improving the independence of individuals with motor disabilities and reducing the burden on the health system. Project Objectives: The overarching aim of VR-BCI4PM is to enhance mobility and independence for individuals with neuromotor impairments by developing and validating a novel training system that combines VR and a hybrid brain-computer interface (hBCI). The project is structured around four specific objectives: 1. Development of an innovative VR simulator (VR-PM) for clinically validated, PMP-aligned training of powered mobility. 2. Design and evaluation of a self-paced hybrid BCI, integrating EEG and EMG signals, to enable joystick-free control for individuals with severe motor impairments. 3. Integration of VR and BCI technologies to allow customizable control modalities tailored to user needs. 4. Clinical validation of the system (VR-BCI4PM) with a representative sample of individuals with various neuromotor disorders. The project will be carried out through three work packages (WPs) across 24 months (+ an additional 6 months). A longitudinal, exploratory clinical study will assess both technical performance and user outcomes. Impact: The VR-BCI4PM platform represents an innovative leap in accessible rehabilitation technology. By enabling effective, user-tailored training for powered mobility, even among those with severe impairments, this system addresses a major clinical and societal need. Its hybrid design will extend the accessibility of VR training beyond current technological limits, making independent mobility a reality for a broader population. Clinically, it will enable therapists to personalize interventions using objective metrics. From a health policy perspective, it offers a scalable, cost-effective alternative to hospital-based training, potentially reducing long-term care dependency and increasing quality of life for thousands of individuals. Results The project, VR-BCI4PM, developed and validated a virtual reality system controlled by hybrid brain-computer interfaces to support powered wheelchair training for people with severe neuromotor disorders. Its main goal was to improve mobility, autonomy, and accessibility for users who may not be able to use standard joystick-based wheelchair controls. The project achieved its main technological and scientific objectives. It produced an integrated platform combining VR wheelchair simulators with multimodal control interfaces based on EEG, EOG, and EMG signals. Three BCI configurations were implemented: an EEG motor-imagery system, a hybrid EEG–EOG system, and a hybrid EOG–EMG system. These allowed users to control the virtual wheelchair using brain activity, eye movements, blinking, or residual muscle activity. Two VR simulators were developed. The first, VR-PMP, was based on the Powered Mobility Program and adapted for pediatric users. It was tested with adults and with children/adolescents with cerebral palsy, including both clinical and home-based sessions. The second, VR-WST, was based on the Wheelchair Skills Test and validated with adult participants. The project also developed an objective measurement system using inertial sensors and embedded processing to quantify wheelchair driving performance more objectively than clinician visual scoring alone. The BCI systems showed promising performance. In able-bodied participants, the EEG/EEG–EOG systems reached around 0.71 accuracy, while in pediatric participants with neuromuscular disabilities the overall accuracy was about 0.67. In successful pediatric sessions, performance improved to about 0.77 accuracy, with sensitivity and specificity also increasing. Importantly, all pediatric participants involved in the final integrated phase were able to control the virtual wheelchair and complete simulator tasks, although effort varied depending on impairment severity. A major result was the project’s contribution to inclusive and user-centered assistive technology. During implementation, the focus shifted more strongly toward pediatric populations, especially children and adolescents with cerebral palsy, because this group represented a clear clinical need and a gap in the literature. The project also prioritized accessibility, co-design, and multimodal interaction over a simple pre–post clinical performance study. The project reached an estimated TRL 4–5, meaning the system was validated in laboratory and relevant environments and is ready for further clinical and translational development. Future work will need larger and more heterogeneous clinical samples, longer-term validation, improved robustness, and further personalization of BCI control, including possible use of explainable AI. The project also produced strong dissemination and training outcomes. Results were presented at national and international conferences, including SIMFER, ISVR, IEEE SMC, GNB, SIRN, I-RIM, and other clinical and engineering events. Several publications are under review, including work on VR-based wheelchair training, physiological monitoring, hybrid BCI protocols, objective measurement systems, and systematic review activity. The project also supported four Master’s theses and contributed to PhD research, involving one PhD student and several post-doctoral researchers. Overall, the project successfully demonstrated that combining virtual reality, hybrid BCI control, wearable physiological monitoring, and objective performance measurement is a feasible and promising approach for future powered mobility training. Its strongest outcomes are the development of inclusive VR-BCI prototypes, successful pediatric validation, objective assessment methods, and a clear pathway toward future clinical adoption and technology transfer.
Project details
Unibo Team Leader: Marilisa Cortesi
Unibo involved Department/s:
Dipartimento di Ingegneria dell'Energia Elettrica e dell'Informazione "Guglielmo Marconi"
Coordinator:
ALMA MATER STUDIORUM - Università di Bologna(Italy)
Total Eu Contribution: Euro (EUR) 200.000,00
Total Unibo Contribution: Euro (EUR) 119.887,00
Project Duration in months: 24
Start Date:
28/09/2023
End Date:
28/02/2026
