1. 1Department of Electrical and Computer Engineering Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz Ian McAlister Colin Smith Chris Neyland Erick Drummond TEAM GANZ Comprehensive Design Review

2. 2Department of Electrical and Computer Engineering OUTLINE  Project Update Summary  CDR Deliverables Review  Final Product Design  Beagle Board  Hardware  Enclosure, Battery, Interface, Lighting  Image Processing  Experimental Design  Extensive testing of chosen algorithms on PC  Implement Chosen Algorithm on Beagle Board  Demonstration  Proposed Demonstration for FPR and SDP Day  Conclusion and Pathway to FPR

3. 3Department of Electrical and Computer Engineering Project Update Summary  Product Requirements  Low Cost  Intuitive Interface Design – 2 to 3 buttons max  Small Form Factor - <50in 3  Battery Life – One Hour Continuous Use  Performance - 90% Accuracy  Upgradable Bill Library

4. 4Department of Electrical and Computer Engineering Project Update Summary  System Specifications: Bill Reading Device  The Bill Reader Device Estimated Specifications:  1 Processing Unit: Beagle Board  Commercial Off-The-Shelf (COTS) Camera: PS3 EYE  Speaker  LED Lights  SD Card  Microcontroller and a few Buttons for User Interface  Low Power Usage  System standby when not in use

5. 5Department of Electrical and Computer Engineering Project Update Summary  MDR Milestones  Beagle Board - Developed Image to support  Webcam Drivers, USB Keyboard, Audio Playback, Implementation of compilers, Open CV Support  Image Processing Algorithms (Description on Page One of Handout)  Autorotation  Template Matching

6. 6Department of Electrical and Computer Engineering CDR Deliverables  Final Product Design  Beagle Board  Hardware  Enclosure, Battery, Interface, Lighting  Image Processing Algorithm  Experimental Design  Extensive testing of chosen algorithms on PC  Implement Chosen Algorithm on BeagleBoard  Conduct time measurements  Demonstration  Current Status and Demonstration

7. 7Department of Electrical and Computer Engineering Final Product Design – Beagle Board  Beagle Board Integration Challenges:  Difficult to Implement Code on Beagle Board  Subtle Differences in OS Caused Issues with Coding  Subtle Differences in Platform Caused Similar Issues  Needed to set up a Fast, Stable, and Intuitive Development Environment  SSH Used to Speed up Testing Cycle  Caused Audio Playback Issues  Significant Time Delay in Template Matching

8. 8Department of Electrical and Computer Engineering Final Product Design – User Interface  Status of Intuitive Interface  Breadboard Setup  Using Atmega168 to Simulate Keyboard  Coding/Debugging in Progress  Need to  Finalize and Back-up the Drive Image  Ensure all Drivers for Auxiliary Devices are Loaded at Startup

9. 9Department of Electrical and Computer Engineering Final Product Design – Hardware  Hardware  Lithium Ion Battery Pack  2 x 3.4V Re-chargeable Batteries 2200mAh  Need to  Disassemble Hub  Finish Intuitive Interface  Complete Breadboard Wiring of auxiliary devices  Research Ideal Speaker  Fabricate Enclosure

10. 10Department of Electrical and Computer Engineering Final Product Design – Hardware Development Model: Production Model: Major Differences: Production model removes Keyboard, Monitor, and Ethernet USB Hub, Microcontroller, and Voltage Regulator will be condensed into a smaller PCB

11. 11Department of Electrical and Computer Engineering Final Product Design – Image Processing  Find Best Match by Using Normalized Cross-Correlation Coefficients Function (NCC)  C = Correlation Matrix  I = Captured Image, = Sample Image Mean at location: u,v  T = Template Image, = Template Mean  u,v = Indices at Point of Correlation  Best Match Retrieved by OpenCV Function: cvMinMaxLoc( image, &minval, &maxval, &minloc, &maxloc, 0 ); Where maxval is a number from -1 to +1 corresponding to best match.  This Value Can then Be Used to Determine if There is a Match

12. 12Department of Electrical and Computer Engineering Experimental Design  Metrics  Time  False Positives  False Negatives  Correct Identification  Variables – Reduced From Change in Design  Degraded Glass Surface  Lighting  Degraded Bill Condition

13. 13Department of Electrical and Computer Engineering Experimental Design Metrics ExperimentTimingFalse PostivesCorrect IDFalse Negatives 1. Ideal Conditions 2. Degraded Glass Surface 2a) Fingerprints 2b) Scratches 2c) Oily type substance 3. Unlevel Surfaces 4. Degraded Bill Condition 4a) Crumpled Bill 4b) Faded Bill 4c) Written on Bill 4d) Missing or Folded Corner 5. Varying Conditions - No Fixed conditions

14. 14Department of Electrical and Computer Engineering Results From Testing  Initial Lighting Bad  Initial LEDs cause reflection and non-uniform illumination  Solution  High Intensity to be located away from Camera  Large Degree of Illumination for uniform lighting  When “Scanned” Image Deviates from Center  Correct Identification Significantly Reduced  Solution  Flip BATBRD Upside Down  Guide Bill using “rails”  BATBRD Upside Down with Rails to Guide User

15. 15Department of Electrical and Computer Engineering Results From Testing (cont.)  Implementation of Image Processing Algorithm on Beagle Board  > 20 seconds With Only Partial Bill Library  Solution  Reducing pixels by a factor of 2 = Reduction of 15 seconds. (Increased reduction may occur).  Remove Autorotation Algorithm  Remove Image Normalization from Algorithm

16. 16Department of Electrical and Computer Engineering Final Prototype  Show a picture of all the devices again only highlight the devices we plan to hack and reduce in size

17. 17Department of Electrical and Computer Engineering Final Prototype  Put the 3D CAD Picture here make sure that we have either recorded rotating or have the program up to rotate it

18. 18Department of Electrical and Computer Engineering Miniaturization of Prototype  Talk about how we plan to reach an eventual wallet sized device

19. 19Department of Electrical and Computer Engineering FPR and SDP Demonstration Proposal  FPR Demonstration  Using Final Prototype including Project Enclosure  Request Evaluators to close eyes and use BATBRD  Scan Different types of Bills  Ideal  Faded or Degraded Bill  Defaced Bill  SDP Day Demonstration  Jenny has volunteered to demonstrate/discuss the BATBRD  Include blind fold for others who wish to test the device

20. 20Department of Electrical and Computer Engineering Conclusion  Testing Revealed Many Issues  Lighting – Solved by Specific Lighting Scheme  Scanned Image Not Centered reducing correct identification  Image Processing on Beagle Board Slower than expected  Reduce Pixel Size  Optimize Algorithm  Abandon Autorotation  Change Design

21. 21Department of Electrical and Computer Engineering Conclusion and Pathway to FPR  Continue optimizing Algorithms to reduce time while ensuring >90% correct identification.  Develop large database of testing results  Disassemble and customize other devices to ensure small form factor  Develop Project Enclosure  Speak with Jenny and Product Developers regarding design and material to use  Ensure Device can still be easily troubleshot once enclosed

22. 22Department of Electrical and Computer Engineering Gantt Chart

23. 23Department of Electrical and Computer Engineering FPR Deliverables  List FPR Deliverables here

24. 24Department of Electrical and Computer Engineering Questions ?