1. Krishna Jharjaria TJ Strzelecki Rick Schuman Matt Waldersen

2.  The Mind Reader is a mobile brain-computer interface.  Computer applications will be presented to the user through commercially available video glasses.  An EOG and commercially available EEG will be mounted inside of a common enclosure and will enable the user to navigate and select various applications.  A dsPIC microcontroller will be used to acquire the EOG and EEG signals, the EEG signals will be analyzed by an FPGA and a BeagleBoard XM will control the virtual reality environment as well as execute all of the computer applications

3. 1) An ability to encode/decode data packets from a NeuroSky EEG. 2) An ability for a user to select applications based on signals from a NeuroSky EEG. 3) An ability for a user to navigate between different applications on a display using EOG signals. 4) An ability for the system to interactively train the user to effectively operate the device. 5) An ability to display a live video stream from an external camera module, and integrate applications into the video system.

5.  Considerations  Signal Processing abilities  Digital Communication pins  Optimized for C compiler  Processing speed  Resources and reference material  DSPIC33EP512MU810  Extensive DSP Library with built in FFT function  4-UART; 4-SPI; 2-I2C  Optimized for C compiler  ~53K of RAM  Large online community

6.  Low Noise  Dual Polarity  Model Available in PSPICE  Multiple Op-Amp Models  Available in DIP and SMD configurations  With exception of the TLV2211  High Accuracy

8.  High Resolution  19.5 Effective Resolution (Bits RMS)  Differential input and 2.5V reference output eliminates need for virtual ground  Integrated Programmable Gain Amplifier  Integrated Digital Filter  Available in both DIP and SMD packages

9.  Considerations  Size (Logic Blocks)  Number of I/O Pins  Built in Functionality  Power Consumption  DLP-HS-FPGA3 USB - FPGA Module  25,344 Logic Blocks (11,264 slices)  63 Available user I/O channels  32 Dedicated Multipliers, libraries for floating point arithmetic  FPGA module contains: regulators, clock generator, SDRAM, USB programmer, SPI Flash, compatible with Xilinx ISE WebPack  FPGA operates on 3.3V, sinks/sources approximately 24mA per I/O pin

10. BeagleBoard-xM  1.0 GHz ARM Cortex-A8 Processor  512 MB RAM  8 GB microSD HDD ▪ libraries and main program  4 USB 2.0 Slots ▪ Mouse/Keyboard dev, Webcam  Expansion Header (Data Transfer) ▪ o SPI ▪ o UART ▪ o I2C ▪ o GPIO

11.  Modular design  Maintains separation of digital circuitry and sensitive analog EOG circuitry  Enables the EOG circuitry to be placed as close possible to signal electrodes  Video glasses will be used in order to ensure that the video display is directly in front of the field of view  Light Weight Single Board Computer

12. BeagleBoard-xM  Size: 3.35” x 3.45”  Weight: < 1 lb. Logitech C310 HD Cam  1280x720p frame capture  < 8 oz. Vuzix Wrap 920  Twin 640x480p Display  < 3 oz.

13. EOG inputs Power Supply Microcontroller Interface FPGA and Beagle Board

14. 11.1V rechargeable Lithium ion Battery 5V convertor For FPGA and Beagle Board 3.3 V convertor For Microcontroller 2 Switch Mode regulators

15. Connected to EOG via SPI Connected to EOG via SPI Logic Level convertor From 5V to 3.3V Registered Jack (RJ11)

16. Oscillator In System Programmer PCB mount Reset Through hole Reset Connections Switch debouncer Pin 13 MCRL Pin 15 PGEC Pin 16 PGED EEG input via UART

17.  Baud rate = 115200 bps  Specified by the NeuroSky MindWave

18.  EEG frequency range 4-7hz  Array size 512  Output of the FFT resides in RAM  Inbuilt FFT API: FFTComplexIP() BitReverseComplex() SquareMagnitudeCplx() 4-7hz

19. Beagle Board FPGA Module Logic Level Convertor From 3.3V to 5V Select button Switch debouncer 4 bit EOG output Up, down, left, right 2x5 bit EOG input 4 I/O for EOG 3 SPI pins for EEG

20.  Main Board  Multiple Voltages needed  Power Traces at least 40 mil, signal traces no smaller than 10 mil  Isolate oscillator  Physical location of connectors  Decoupling Capacitors  EOG Board  Sensitive analog circuitry  Distance between electrodes and filtering/amplification  Physical size of EOG PCB

30.  Electrooculography measures the signal given off of corneo-retinal potentials in the eyes  An EOG circuit consists of an instrumentation amplifier, amplifiers, low pass filtering and high pass filtering  A high pass filter is used to eliminate a naturally occurring DC drift  A low pass filter is used to remove all other external noise

31. Instrumentation Amplifier  High Input Impedance  Includes both a high pass filter to eliminate DC drift and a second order low pass filter

33. Sallen-Key 2 nd -Order Low Pass Filter  Eliminates noise from external sources

35. First Order Low Pass Filter – Amplifier Circuit  Eliminates noise as well as amplifies the signal

42. Voltage (V) Time (s)

43. Voltage (V) Time (s)

44. ∆Voltage Sample Number

45.  What is an Artificial Neural Network?  Simplified mathematical model of the human brain  Utilizes a network of “neurons” to model relationships between inputs and outputs  How does a ANN work?  ANN implement non-linear functions in each neuron  This function sums weighted input values and passes them through a non-linear function, usually a Sigmoid function  This output then propagates to further network layers for the process to be repeated  Why use an FPGA implementation  Highly parallel algorithm, with transistor like output  Need for quick, complex calculations

47.  Basics of A Neuron  Each neuron contains a number of synapses, one for each previous layer neuron connected  A synapses will take inputs and multiply it by a predetermined specific weight  The weighted values will then be accumulated and funneled into a nonlinear activation function (Sigmoid Function)  Hardware Design Considerations  Need multiple multipliers, one for each synapses (38 multipliers) ▪ Need to condense the size of a neuron ▪ Using control logic and multiple clock cycles we can lower the required multipliers to a single multiplier per neuron (12 multipliers)  Implementation of non-linear Sigmoid Function ▪ Look-up Table

49.  Where we proceed from here  Training and testing of preliminary ANN using Matlab ANN toolbox ▪ This will allow the weighted values to be hardcoded into the FPGA  Design and testing of single neuron in VHDL  Development of a proven working topology

50. Tentative HUD Features Webcam Live Feed EEG Signal Data display Home Screen 3 x 3 application layout Concentration to select app Programs Minimal key input Closed apps return to Home

52.  October 15-19  PCB Proof of Parts  October 22-26  Simple Software Suite  Sample EOG circuit  ADC microcontroller communication  October 29 – Nov 2  Micro to Single-board communication  EOG ANN finished in MATLAB  Nov 5 – Nov 9  Live Feed Visual Software Suite  EEG ANN prototyped in MATLAB  PCB assembled  Nov 12 – 16  EOG ANN finished on FPGA  Nov 19 – 23  Software Test Application  User Training Application  Nov 26 – Nov 30  Troubleshoot  Dec 3 – Dec 7  Project Complete