1. Progress Presentation IRALAR Breanna Heidenburg -- Michael Lenisa -- Daniel Wentzel Advisor: Dr. Malinowski

2.  Project Overview  Application Layout & Progress ◦ Image Processing ◦ Calibration ◦ Clicking ◦ Communication ◦ User Interface  Overall Progress

3. What is our project?

4.  Track a user’s eye ◦ Small form factor camera  Use input to control cursor ◦ Where you look is where the cursor is ◦ Need to calibrate for every user  Create user interface

5. How will we achieve our goal?

7.  Hardware ◦ Camera  QuickCam Pro for Notebooks ◦ Polarizer  Tiffen 25mm polarizing filter  Removes glare from eye reflections ◦ Lighting  Diffuse LEDs  Not sure if they are necessary, will find out when we have the display

8.  New Image processing methods: ◦ Blob Recognition ◦ Classic Image Processing Techniques  Contrast stretching  Thresholding  Adaptive Windowing

9.  Sample Recognitions: Difficult Low Light False Positive

10.  Why do we need to calibrate? ◦ Map the user’s eye to a point on the screen ◦ Everyone’s eyes are different ◦ Account for various positions of the camera  What does our data look like? ◦ Somewhat unreliable ◦ Problem:  Movements while taking data

11.  X-Axis values

12.  Y-Axis values

13.  What does the data tell us? ◦ A 2D mapping is not good enough ◦ Horizontal eye movement is closer to linear than vertical eye movement ◦ Vertical eye movement depends on both coordinates of data

14.  How do we solve the problem? ◦ Multiple Variable Linear Regression – Least Squares Y = B 0 + B 1 x 1 + … + B k x k ◦ Uses matrix algebra to obtain a coefficient matrix B[] = (X’X) -1 X’Y

15.  Results ◦ pixelX = 224.9 + 4.8*eyeX – 2.9*eyeY ◦ pixelY = 1612.5 + 2.0*eyeX – 5.6*eyeY ◦ Error = very high (need more than 4 random data points) B[] = (X’X) -1 X’Y

16.  How do we click? ◦ Monitor eye movements ◦ Identify pauses ◦ When eye position is within a small percentage for a certain amount of time ◦ Set to 5 frames (~200 mS) ◦ Generally, it takes 230 mS for a hand to click a mouse.

18.  Why is it important? ◦ Allows multiple processes to communicate ◦ Allows relay of time sensitive information  UDP vs. TCP ◦ UDP -> ‘Connectionless’ ◦ TCP -> ‘Connection oriented’  Multi-threading ◦ Necessary for running multiple pieces of code in a single process

19.  Multi-threading ◦ Issues with public variable usage ◦ Solutions we are using:  Critical Section  Raises thread priority (does not allow for interrupts)  Mutual Exclusion  Creates a variable lock  Using variable first requires ‘unlocking’ it

20.  Achieved via a combination of Hardware and Software  Software: WPF & C# ◦ XAML (eXtensible Application Markup Language) ◦ Very similar to XML ◦ Uses ‘code-behind’ in a similar style to javascript  ‘code-behind’ is in C#

21.  Framework is in place for all pages  Navigation between pages is complete  Core operating portions are complete

22.  Hardware ◦ LitEye LE500 has been ordered (will arrive early-mid March) ◦ SVGA resolution (800x600) ◦ Transparent

23. How far along are we?

24.  HMD has arrived (today) ◦ Have not had much time to work with it ◦ We aren’t 100% sure how we will physically mount the camera to the display ◦ Aren’t sure if extra lighting will be necessary

25.  Image Processing ◦ Still working on some recognition improvements  Calibration ◦ Minor adjustments left to be made  Communication ◦ Minor adjustments left to be made  User Interface ◦ Only essential parts are included at the moment Still ironing out details on what will be included in final version