Every year, several thousand severe traumatic brain injury cases occur, e.g., through accidents or stroke, often resulting in permanent partial loss of consciousness and communication ability. Misdiagnosis rates of DoC patients are high (~40%), and there are limited means for therapy so far. NeuroCommTrainer is a multi-modal brain-computer interface for such minimally responsive individuals, using the senses of touch and hearing together with event-related neural responses or residual motor-function to re-establish basic communication, e.g, to allow yes/no answers, or operation of a speller for text input.

In contrast to existing systems such as Mindbeagle, NCT is based on modular, adaptable wearables, uses individualized BCI machine learning, aims to support both P300-based communication as well as training of residual motor function, and automatically identifies periods of highest patient wakefulness for training.

Requirements

We were tasked to build wearable components for a modular system, suitable for the clinical environment the users live in. Each patient has different abilities and a modular system is needed to accommodate sensing and actuating modalities appropriate to each case. Donning these wearables on the user, even in cases of spastic muscles, needs to be as easy as possible, as well as quick removal in medical emergencies. Motion sensors and tactile actuators need to be placed on hands and/or limbs; a controller unit can hang from the bed or an infusion stand. Although the patients are in a state of unknown consciousness, they participate in activities such as ergotherapy, and thus the system needs to be portable and easily to handle by clinical staff when patients are bedded or seated. On-patient wired interconnects are acceptable, but the system components that come into contact with the user need to be galvanically isolated from power mains supply. Consequently, the on-patient controller in our system is to communicate wirelessly and be battery-powered with a day (>=8 h) of operation. Latency and jitter incurred by wireless communication have to be compensated to allow correlation with high-frequency EEG signals.

Components

The NCT system, consists of a stationary controller, novel flexible EEG electrodes (see references below)) on wireless amplifiers, a portable wireless controller on the user’s bed, and patient-adaptable sensor-actor wearables. The portable/wearable system components were built using BRIX5. The Arduino-based nature of the BRIX5 base controller allowed non-engineering experim Sensors and tactors can be chosen to match the patient’s needs.

• On the user’s bed/chair: one NeurCore, wireless relay between BCI controller and the following wearables:
• On legs and arms: up to six motion sensors and up to four Lofelt L5 tactors.
• On hands: two vibrotactile (VT) gloves.

NeuroCore: Wireless Sensor-Actor Controller

The backbone of the portable system is a controller we call NeuroCore that interfaces between the central BCI controller and sensors/actors. The NeuroCore hangs on the patient bed/seat, and basic functions can be controlled through an display and buttons. Tactile stimuli are either rendered through an integrated Audio-400 module on up to four Lofelt tactors, and/or the VT gloves.

NeuroCore’s is built around a Raspberry Pi 3A+ SBC, chosen for its compact size, sufficient computational power, and good software support. A SparkFun Qwiic-Hat and a Smart Prototyping Qwiic Mux allow connection of up to six BNO sensor modules and the two gloves. A 3-cell 18650 LiIon battery provides charge for >10 hours of operation, a ZIO Battery LiPo Battery Manager handles power supply, charge control and monitoring. An OLED displays the system status and buttons give control over basic system functions to developers and experimenters. The NeuroCore is housed compactly in a 3D-printed case (14 x 7 x 2.5 cm³) and extensions are connected through long (1-3 m), shielded Qwiic cables.

The NeuroCore and the BCI controller communicate through Wifi over the LabStreaming Layer (LSL) protocol, which provides the required timestamping and clock synchronization to mitigate transfer delay and jitter. Separate LSL streams provide sensor data, system status, and stimuli timing information to the BCI controller, which in turn requests haptic stimuli. The sensor readings for 4 BNOs stream at ~80 Hz (gravity compensated acceleration and angular velocity, 3 axes each).

Hands: Vibrotactile Glove

For tactile stimuli rendered to the hands, we developed custom sensor-actuator gloves, based on archery gloves. They fit on varying hand sizes and can be easily donned on subject’s hands by experimenters. Embedded into the index, middle and ring fingertips are LRAs that are controlled by a QuadDRV module attached on the back of the hand. Tubular wire housing physically protects the fragile actuator lead wires.

The gloves can either operate standalone or as part of the NCT controller. Standalone, the central controller is a BRIX5 base module attached on the back of the wrist, connecting to a experiment control computer through USB. A study on evaluating gloves for tactile P300 using the standalone setup and the novel EEG electrode grids was published by Blum et al..

Lessons Learned

BRIX5 and the open prototyping tools available today flexibly supported a fast-paced, interdisciplinary prototyping process. NeuroCore and Gloves could be prototyped and tested rapidly, for example, NeuroCore’s base functionality was established and tested within two weeks with two developers. The ability to use parts of the system as standalone instrumentation enabled individual evaluation and side projects. The Arduino-based nature of the BRIX5 base controller allowed non-engineering experimenters to modify firmware details on their own, such as adjusting stimulus parameters. The universal Qwiic/JST SH connectors made replacing cables for adapting the system to different users and applications easy and fast.

AmbiSense

We developed a modular ambient sensing platform to support the ml-based wakefulness-detection, called AmbiSense.

Vibromaster 8000

Supporting a related research project on vibrotactile BCIs for individuals with Amyotrophic lateral sclerosis (ALS), we developped a eight-channel controller for eccentric rotating mass tactors, called VibroMaster 8000.

Publications