Augmented Cognition International, San Francisco, October 16, 2006 Novel Hybrid Sensors for Unobtrusive Recording of Human Biopotentials Robert Matthews, Neil J. McDonald, Harini Anumula, and Leonard J. Trejo Quantum Applied Science and Research

QUASAR Breakthroughs in Sensor Technology for Applications from Minerals to Medicine

Our Vision for Augmented Cognition Truly deployable unobtrusive biosensors –Nearly invisible to the user –Seamlessly integrated with clothing and equipment End-to-end systems for operational biosensing –Broad application spectrum (e.g., EEG, EOG, ECG and EMG) –Low-power, long lasting, wireless electronics –Robust algorithms tolerant of noise and sensor dropout –Intelligent systems adapt for situational or individual differences Focus of this presentation: Hybrid EEG Sensors NO ADHESIVES NO WIRES NO PREP NO GELS

Barriers to Truly Deployable EEG Sensors Large common mode signals Frequent user motion will generate signal artifacts Sensor harness physicality issues –Lack of user compliance –Possible incorrect application of sensors –Fitting sensors into wearable garments –Variability over subjects –Hygiene Battery power issues Size and weight issues Long-term usability and reliability

QUASAR EEG Sensor Technologies QUASAR Capacitive sensors (on skin EEG, EOG, ECG) QUASAR Next Gen sensors (standoff detection) DARPA AUGCOG AUGCOG, Aberdeen, MEMS EX 0405/06 06/07 First Test

QUASAR EEG Sensor Technologies QUASAR Capacitive sensors (on skin EEG, EOG, ECG) QUASAR Next Gen sensors (standoff detection) QUASAR Hybrid sensors (through hair) DARPA AUGCOG AUGCOG, Aberdeen, MEMS EX DARPA Aberdeen Prototype Availability 0405/06 06/07 First Test

Pros and Cons of Current EEG Sensors Conductive sensors –Generally good signal quality, low noise –Most require some skin preparation –Produce skin irritation and risk of infection –Micro- or nano- needle sensors break off and remain in skin –Cannot be worn for extended periods of time (electrode drying) Capacitive sensors –Sensitive to static charges, stray E-fields, and motion artifacts –No skin preparation, electrolyte drying or skin irritation

Hybrid EEG Biosensors (the best of both worlds) Capacitive QUASAR (IBE) EEG sensors –Presently record high-quality EEG signals on bare skin –Through-hair EEG sensing is not currently feasible –Through-hair will be feasible with next-gen technology Advanced QUASAR capacitive sensors systems make new hybrid EEG sensor technology possible –Hybrid sensors use ultra high resistive contact in conjunction with capacitive contact –Hybrid sensors do not modify or invade the skin and consequently do not cause skin irritation –Hybrid sensors are a new option for wearable EEG arrays

Deployable EEG/EOG Sensor Concepts QUASAR Hybrid or MEMS/Capacitive Electrodes Audio Headset EOG/EEG Glasses QUASAR Capacitive Electrodes

Initial Hybrid Bioelectrode Demo (3/2006) QUASAR and wet electrode in the C z position – filtered in 5-15 Hz bandwidth Test set up – standard electrode positions Correlation between two sensors Trial for an outside entity Sensor used (old) New sensor

Four HCI contexts tested with side-by-side “wet” electrodes and QUASAR Hybrid EEG sensors 1.Baseline: Eyes Open/Closed (desynchronization of EEG alpha rhythm) –30 s open / 30 s closed; repeated 3x over session 2.Real/Imaginary Button-press (desynchronization of EEG  -Rhythm) –Reach and press key on keyboard; repeat with imaginary motion 3.Visual Oddball (modulation of P300 ERP component) –Count the number of visual targets (row or column of matrix) 4.Memory/Cognition (modulation of EEG alpha and theta rhythms) –Count backwards from 100 by ones or sevens for 60 s All conditions: recorded Fz, C3, Cz, C4, referred to right ear Ag-AgCl disc electrodes, QUASAR IBE, and hybrid sensors

EEG Eyes Open/closed: Correlation of Signals over 30-s Long Segments Eyes-open EEG task, signals normalized to zero mean and unit standard deviation, displaced vertically to allow comparisons. From the top down: QUASAR Cz, wet Cz, QUASAR Fz, wet Fz, QUASAR C3, wet C3, EOG, and ECG. Red and blue lines are for the QUASAR sensor and wet electrode recordings, respectively.

EEG Eyes Open/closed: Distribution of Correlation Coefficients for 1-s Segments Histogram of correlations over consecutive 30 1-s segments for QUASAR Fz - wet Fz. In 26/30 segments the QUASAR-wet correlation was 0.80 or higher. EEG signal band-limited to 0-40 Hz.

EEG Eyes Open/closed: Correlations of Power Spectra. Power spectral density functions of 30-s EEG recordings during eyes-closed session and following eyes-open session (Welch’s method, window = 2 s, overlap = 1 s, FFT length = 1024, sampling rate = 100 Hz). QUASAR-Wet Correlations Eyes closed: Eyes open:

Button-press Tasks: Sample Power Spectra (Hybrid Sensors only). QUASAR EEG sensor: PSDs of 60-s EEG recordings at Cz during button-press task, imaginary button-press task and a baseline eyes-open session (30-s) immediately before.

Counting tasks: Sample Power Spectra PSDs of 30-s EEG recordings during easy and hard levels of counting task. Easy-hard spectral differences appear in the theta (4-7), alpha (8-12), and SMR (14-16 Hz) bands of the EEG spectrum. Inter-sensor correlations between QUASAR and wet sensors at Cz in the easy and hard conditions were and , respectively. r =.9993 r =.9987

Visual Oddball Task: Sample ERP Averages Oddball (P300) Task: count deviant stimulus (row or column; P = 1/7) Post-stimulus Time (ms) Average ERP Amplitude (  V)

QUASAR hybrid sensor EEG signals correlate extremely well with simultaneously recorded wet electrodes. –QUASAR – wet sensor correlations in the time domain were typically at 0.8 or better (interchannel correlations of wet sensors ranged from ) –Correlations of QUASAR - wet frequency spectra within the EEG band of 0-40 Hz were typically above QUASAR hybrid sensors can gauge EEG important for AugCog & HCI –modulation of EEG rhythms, such as alpha, theta, and SMR –detection of P300 ERPs for discrete stimuli Conclusions from Sensor Tests

Development of an Integrated Neurocognitive Sensor Array Through clothing ECG On skin EOG On skin EEG Through hair EEG sensor Algorithms to classify even in noisy environments Fully deployable Bio monitoring system Data Acquisition Sensor Suite AUGCOG & others Aberdeen SBIR Fully Deployable Bio monitoring system Unfunded Sensor Harness Phase II program

Overview of Two System Concepts Zero preparation time –No gels or electrode settling time –No consumables –Simple to put on/take off Wireless communication to processing unit allows freedom of movement and reduces artifacts C2 type environment Dismounted Infantry

10-Channel EEG Prototype DAQ Module 10 channel EEG Ultra low power (2 days on 2 AA batteries) Over 80dB common mode rejection Up to 1kHz sample rate Advanced Wireless A/D System

Wireless Device uC + Radio (nRF24E1) 50 ohm Matching Network Regulator (LTC ) Chip Antenna (AN9520) Dual MOSFET (NTHD4508N ) Tri-Axial Accelerometer + Temp Sensor (H34C) Interface Flex. PCB High bandwidth device low power optimized for duty cycling

Algorithm Development* Irrelevant probe ERP-based workload monitor On-line classifier using adaptive KPLS feature extraction and adaptive classification networks −Can adapt for individual or situational differences in features and feature mappings −Applications tested: − On-line cognitive fatigue assessment (3.5 s update rate) − Multimodal real-time advisory system for human-automation reliability (2 s update rate) − Brain-computer interface (2 s update rate) Basic advances in signal processing (denoising, kernel methods, software/hardware artifact removal) *QUASAR & Partners: NASA, Pacific Development & Technology

Real-time EEG/Mental State Classifier

EEG-based Classification with Low-density Arrays or Low-SNR Signals Robust algorithm applied to mental fatigue classification using multichannel EEG spectra Trained KPLS-DLR classifier in Air Force pilots using 19 electrodes Corrupted signal from SNR of 0 dB to -18 dB Reduced density from 19 to 4 electrodes Preserved >80% accuracy with -9 dB SNR and four electrodes

Multimodal Assessment of Cognitive Overload US Army (Phase II SBIR) Dismounted warfighter simulation EEG/ERP/ECG-based prediction of mental overload Individual models, toxin effects EEG Probe ERP ECG Cognitive overload algorithm Real-time alert or advisory signal KPLS-DLR classification of 20-ch EEG spectra from 2-s epochs