1. 12. General Clinical Issues Relevant to Brain-Computer Interfaces 2011.4.12. YOON, HEENAM

2. Your slogan Contents 1. Introduction 2. Definition and Essential Features 3. General Clinical Considerations of BCI 4. Current BCI Platforms 5. Conclusion

3. Your slogan Introduction 1. Emerged Idea - Machines that can be controlled by one’s thoughts - Machines that can be controlled by one’s thoughts - Machines that create a new output channel from the - Machines that create a new output channel from the brain beyond the natural motor and hormonal brain beyond the natural motor and hormonal commands commands 2. Terms - BCI (Brain Computer Interface) - BCI (Brain Computer Interface) - DBI (Direct Brain Interface) - DBI (Direct Brain Interface) - BMI (Brain machine Interface) - BMI (Brain machine Interface) - Motor Neuroprosthetics - Motor Neuroprosthetics

4. Your slogan Introduction 3. Source 3. Source - Electrophysiological signal in the brain - Electrophysiological signal in the brain 4. Purposes 4. Purposes - Communication - Communication - Control something - Control something 5. Background 5. Background - Improving the quality of life for people with Motor - Improving the quality of life for people with Motor impairment, including SCI, neuromuscular disorders impairment, including SCI, neuromuscular disorders

5. Your slogan Introduction 6. Environment - Improved insight into the electro-physiological - Improved insight into the electro-physiological principles principles - Development of computer processing - Development of computer processing - Growing social awareness of the needs of the - Growing social awareness of the needs of the severely handicapped severely handicapped

6. Your slogan Introduction 7. Requirement 7. Requirement - Framework of how these systems operate - Framework of how these systems operate - The current BCI platforms and limitations - The current BCI platforms and limitations - The relevant issues when applied clinically - The relevant issues when applied clinically - Milestone for their evolution toward entering - Milestone for their evolution toward entering standard clinical practice standard clinical practice

7. Your slogan Definition and Essential Features 1. Definition 1. Definition - A communication system that does not depend on - A communication system that does not depend on the brain’s normal output pathways of peripheral the brain’s normal output pathways of peripheral nerves and muscles nerves and muscles (The first International Meeting on Brain Computer (The first International Meeting on Brain Computer Interface Technology(2000)) Interface Technology(2000)) - Machine that can decode human intent from brain - Machine that can decode human intent from brain signal alone signal alone · Not required the assistance of motor activity · Not required the assistance of motor activity · A BCI allows for a completely novel output pathway · A BCI allows for a completely novel output pathway from the brain from the brain

8. Your slogan Definition and Essential Features · BCI changes signals from mere reflections of CNS · BCI changes signals from mere reflections of CNS activity (EEG rhythms, neuronal firing rate)into the activity (EEG rhythms, neuronal firing rate)into the intended products of that activity intended products of that activity · Two BCI Systems · Two BCI Systems Input BCI(perception) Input BCI(perception) Output BCI Output BCI

9. Your slogan Definition and Essential Features · Feedback · Feedback - To improve the performance - To improve the performance - A certain level of training and learning is need for - A certain level of training and learning is need for both for the user and the computer both for the user and the computer Hardware Software Electro- physiological Signal Action BCI Nerves Muscles Movements

10. Your slogan Definition and Essential Features 2. Essential elements for BCI - Signal acquisition - Signal acquisition - Signal processing - Signal processing - Device output - Device output - Operating protocol - Operating protocol Feedback

11. Your slogan Definition and Essential Features (1)Signal acquisition - A measurement of the electrophysiological state of the - A measurement of the electrophysiological state of the brain in real time brain in real time - Usually recorded with electrodes(non/invasive) - Usually recorded with electrodes(non/invasive) · EEG: recorded from the scalp · EEG: recorded from the scalp · ECoG: recorded beneath the skull · ECoG: recorded beneath the skull · Single-Unit: recorded by microelectrodes, · Single-Unit: recorded by microelectrodes, individual neuron action potential firing individual neuron action potential firing

12. Your slogan Definition and Essential Features - Other types - Other types · MEG, fMRI, PET · MEG, fMRI, PET -> Costs, slow time constants -> Costs, slow time constants

13. Your slogan Definition and Essential Features (2) Signal processing - Feature extraction & Signal translation - Feature extraction & Signal translation · To extract significant identifiable information · To extract significant identifiable information from the gross signal from the gross signal · To render identifiable information into machine · To render identifiable information into machine Hardware Software Statistical Analysis Meaningful signal Raw Signal

14. Your slogan Definition and Essential Features (3) Device Output - Overt command or control functions that are - Overt command or control functions that are administrated by the BCI system administrated by the BCI system (4) Operating protocol - Important for clinical application - Important for clinical application - refers to the method by which the user controls how the - refers to the method by which the user controls how the system functions system functions - Ex. on/off system, controlling what kind of feedback - Ex. on/off system, controlling what kind of feedback occurs and how fast it is provided manipulating how occurs and how fast it is provided manipulating how quickly system implements command and switching quickly system implements command and switching between various device outputs between various device outputs

15. Your slogan General Clinical Considerations of BCI 1. Safety 2. Durability 3. Reliability 4. Complexity 5. Suitability 6. Efficacy

16. Your slogan General Clinical Considerations of BCI 1. Safety, Durability, Reliability - Consideration for risk - Consideration for risk · Implant must be reliable and durable in its ability to · Implant must be reliable and durable in its ability to acquire signals(DBS, cortical stimulators, grid electrodes) acquire signals(DBS, cortical stimulators, grid electrodes) · Affected by device design · Affected by device design · Patients responds · Patients responds - Duration problems - Duration problems · Cause removal and replacement · Cause removal and replacement

17. Your slogan General Clinical Considerations of BCI 2. Complexity There are factors related to performance that must be deliberated for a BCI to have practical application. These issues include complexity of control and levels of speed and accuracy There are factors related to performance that must be deliberated for a BCI to have practical application. These issues include complexity of control and levels of speed and accuracy - Degree of freedom: dimension - Degree of freedom: dimension ->motor impaired individual ->motor impaired individual · 1DOF: Yes no/Left Right · 1DOF: Yes no/Left Right · 2DOF: x-axis, y-axis · 2DOF: x-axis, y-axis · 3DOF 3 dimensional movements or 2 dimensions +switch · 3DOF 3 dimensional movements or 2 dimensions +switch - For complex movement, more DOF is necessary - For complex movement, more DOF is necessary ->Ex) 7DOF: controlling prosthetic leg ->Ex) 7DOF: controlling prosthetic leg Video Video Limited

18. Your slogan General Clinical Considerations of BCI - Speed and accuracy important in real world scenarios - Speed and accuracy important in real world scenarios (for minimum error) (for minimum error) · Rate of information communicated per the unit time · Rate of information communicated per the unit time · Bit rate ↑ -> Accuracy ↑ · Bit rate ↑ -> Accuracy ↑

19. Your slogan General Clinical Considerations of BCI 4. Suitability - Patients with SCI, ALS and so on, require different devices - Patients with SCI, ALS and so on, require different devices - It is vital that the patient population and its underlying - It is vital that the patient population and its underlying pathology be taken into consideration for what type of pathology be taken into consideration for what type of platform maybe used and what function is provides platform maybe used and what function is provides

20. Your slogan General Clinical Considerations of BCI 5. Efficacy - Example - Example · 1971s · 1971s Degree of control using firing rate of a single neuron Degree of control using firing rate of a single neuron · 1991s · 1991s Applied to human BCI Applied to human BCI - Need of sufficient research to apply to human BCI - Need of sufficient research to apply to human BCI

21. Your slogan Current BCI Platforms  Three types of platforms that have potential for clinical application in the near future EEG-Based EEG-Based ECoG-Based ECoG-Based Single-Unit Based Single-Unit Based  Each has technical aspects that give them dis/advantages regarding their utility in a clinical setting

22. Your slogan Current BCI Platforms 1. EEG-Based Systems - BCI experience has been limited to EEG recordings - BCI experience has been limited to EEG recordings - Studies have investigated the use of sensorimotor - Studies have investigated the use of sensorimotor rhythms, slow cortical potentials and P300 evoked rhythms, slow cortical potentials and P300 evoked potentials potentials

23. Your slogan Current BCI Platforms (1) Sensorimotor Cortex Rhythms (1) Sensorimotor Cortex Rhythms 1) mu rhythm 1) mu rhythm - 8~12Hz - 8~12Hz - Not being involved with motor planning or action - Not being involved with motor planning or action - “rest” activity - “rest” activity - Produced by thalamocortical circuits - Produced by thalamocortical circuits

24. Your slogan Current BCI Platforms 2) beta rhythm 2) beta rhythm - 18~26Hz - 18~26Hz - Normal waking consciousness - Normal waking consciousness · Both real and imagined motor movements are · Both real and imagined motor movements are accompanied by a decrease in mu and beta activity accompanied by a decrease in mu and beta activity over sensorimotor cortex over sensorimotor cortex · BCI doesn’t require actual movement but can be · BCI doesn’t require actual movement but can be accomplished with imagined movement alone accomplished with imagined movement alone · Frequency alterations can occur independently of · Frequency alterations can occur independently of activity in the brain’s normal output channels of activity in the brain’s normal output channels of peripheral nerves and muscles and could therefore peripheral nerves and muscles and could therefore serves as the basis for a BCI serves as the basis for a BCI

25. Your slogan Current BCI Platforms (2) Slow Cortical Potentials(SCPs) (2) Slow Cortical Potentials(SCPs) - Slow changes in EEG potentials centered at the - Slow changes in EEG potentials centered at the top of the head, or vertex, and occur over time top of the head, or vertex, and occur over time scales of several seconds scales of several seconds - Negative SCP: Movement and cognitive functions - Negative SCP: Movement and cognitive functions involving cortical activation involving cortical activation - Positive SCPs: A reduction in such activations - Positive SCPs: A reduction in such activations - Can be learn to control SCP amplitude - Can be learn to control SCP amplitude

26. Your slogan Current BCI Platforms (3) P300 Evoked Potentials (3) P300 Evoked Potentials - When infrequent or significant auditory, visual, - When infrequent or significant auditory, visual, or somatosensory stimuli or somatosensory stimuli - Positive potential in the EEG that peaks at about - Positive potential in the EEG that peaks at about 300ms 300ms - Centered over parietal cortex - Centered over parietal cortex - Depends mainly on whether the user wishes to it - Depends mainly on whether the user wishes to it - Application: Word Processing - Application: Word Processing Video x2 Video x2

27. Your slogan Current BCI Platforms (4) Advantages (4) Advantages - EEG-based paradigms are noninvasive - EEG-based paradigms are noninvasive - Users can control devices without a neuromuscular - Users can control devices without a neuromuscular requirement requirement (5) Disadvantages (5) Disadvantages - Artifact problems(Electrodes movement, muscle - Artifact problems(Electrodes movement, muscle movements, electrical environment) movements, electrical environment) - Signals are distant from the sources - Signals are distant from the sources · Spatial specificity ↓ · Spatial specificity ↓ · Limited frequency detection(~40Hz) · Limited frequency detection(~40Hz) - Prolonged user training time - Prolonged user training time - Electrodes placed in a cap or fixed to the skin - Electrodes placed in a cap or fixed to the skin · Long-term solution ↓ · Long-term solution ↓ prohibition of the complexity of movements

28. Your slogan Current BCI Platforms 3. ECoG-Based Systems 3. ECoG-Based Systems (1) Definition (1) Definition - A measure of the electrical activity of the brain taken - A measure of the electrical activity of the brain taken from beneath the skull from beneath the skull (2) Due to the limited access of subjects, it has not (2) Due to the limited access of subjects, it has not been studied extensively until recently been studied extensively until recently (3) Based on previous EEG based BCI and the (3) Based on previous EEG based BCI and the associated understanding of sensorimotor rhythms associated understanding of sensorimotor rhythms

29. Your slogan Current BCI Platforms (5) Signals are different from EEG (5) Signals are different from EEG - Much more robust - Much more robust · Magnitude is five times larger · Magnitude is five times larger · Resolution is finer · Resolution is finer · Frequency bandwidth is significantly higher · Frequency bandwidth is significantly higher ->High frequency bandwidths carry highly specific and ->High frequency bandwidths carry highly specific and anatomically focal information about cortical anatomically focal information about cortical processing processing (LF: produced by cortical circuits, (LF: produced by cortical circuits, HF: produced by smaller cortical assemblies) HF: produced by smaller cortical assemblies) ->capacitance of cell membranes of the overlying tissue, ->capacitance of cell membranes of the overlying tissue, combined with the intrinsic electrical resistance of combined with the intrinsic electrical resistance of skull, constitutes a LPF eliminating HF skull, constitutes a LPF eliminating HF ->So, HF signals are more prominent when recorded by ->So, HF signals are more prominent when recorded by sub/epidural electrodes sub/epidural electrodes

30. Your slogan Current BCI Platfo (6) Why High Frequency? - Use of power of signal - Use of power of signal

31. Your slogan Current BCI Platforms (6) Research 1) In 2004 1) In 2004 - First use of ECoG in closed-loop control - First use of ECoG in closed-loop control - Brief training(3~24min) 74~100% success rates in - Brief training(3~24min) 74~100% success rates in one-dimensional closed-loop control tasks one-dimensional closed-loop control tasks - Up to 180Hz reflected the direction of two-dimensional joystick - Up to 180Hz reflected the direction of two-dimensional joystick movements movements 2) In 2006 - Demonstrated evidence that ECoG control using signal form the - Demonstrated evidence that ECoG control using signal form the epidural space was also possible epidural space was also possible video

32. Your slogan Current BCI Platforms (7) Durability of subdural-based devices (8)In clinical studies, the use of subdural electrodes as implants for motor cortex stimulation have been implants for motor cortex stimulation have been shown to be stable and effective implants for the shown to be stable and effective implants for the treatment of chronic pain treatment of chronic pain (9)Long-term subdural electrode monitoring for seizure identification and abortion has also been shown to identification and abortion has also been shown to be stable be stable

33. Your slogan Current BCI Platforms (10)Advantages - Higher spatial resolution - Higher spatial resolution - Better SNR - Better SNR - Wider frequency range - Wider frequency range - Lesser training requirements - Lesser training requirements - Lower technical difficulty - Lower technical difficulty - Lower clinical risk - Lower clinical risk - Long-term stability - Long-term stability

34. Your slogan Current BCI Platforms 4. Single-Unit Based Systems (1) Basic Theory (1) Basic Theory Record of firing rate (2) History (2) History 1)1970s 1)1970s - Animals could be trained to modulate the activity of a single - Animals could be trained to modulate the activity of a single neuron in their brains ->one dimensional control neuron in their brains ->one dimensional control

35. Your slogan Current BCI Platforms 2)1980s 2)1980s - Method of decoding three-dimensional hand movement - Method of decoding three-dimensional hand movement direction from a population of neurons in primary motor direction from a population of neurons in primary motor cortex of nonhuman(Ex. Monkey) cortex of nonhuman(Ex. Monkey) - recording the single-unit activity from 50~200 - recording the single-unit activity from 50~200 individual neurons during repeated researching tasks individual neurons during repeated researching tasks 3) Early 1990s 3) Early 1990s - Enhanced neural decoding methods - Enhanced neural decoding methods 4) Late 1990s 4) Late 1990s - Real-time recording of multiple single units - Real-time recording of multiple single units 5) Early 2000s 5) Early 2000s - Recording chronic, single-unit action potential from a number - Recording chronic, single-unit action potential from a number of neurons simultaneously of neurons simultaneously

36. Your slogan Current BCI Platforms (3) Applied to quadriplegic subjects (3) Applied to quadriplegic subjects (4) Limited and preclude making definitive conclusions (4) Limited and preclude making definitive conclusions (5) Two modality tests (5) Two modality tests - Use of “neutrophic electrode” (1998), implanted in 2005 - Use of “neutrophic electrode” (1998), implanted in 2005 - Use of electrode arrays that simultaneously monitor tens - Use of electrode arrays that simultaneously monitor tens to a hundred to a hundred - cortical motor neurons(Utah Array)->implanted in a - cortical motor neurons(Utah Array)->implanted in a spinal-injured quadriplegic patients spinal-injured quadriplegic patients Video

37. Your slogan Current BCI Platforms (6) Life span of microelectrodes ↓ - Need of regular replacement - Need of regular replacement

38. Your slogan Current BCI Platforms

39. Your slogan Conclusion -The field of neuroprosthetics is in its infancy -Current research · E lectrical encoding the information in a human’s thoughts · E lectrical encoding the information in a human’s thoughts -However it can be used by impaired users to interact with his/her environment with his/her environment -Given the rapid progression of technologies, signal analysis tech, emerging ideas should be achieved analysis tech, emerging ideas should be achieved · EEG, ECoG->complexity of control · EEG, ECoG->complexity of control · Single-unit platform->implant durability · Single-unit platform->implant durability · clinical application · clinical application