1. Real-time Computer Vision with Scanning N-Tuple Grids Simon Lucas Computer Science Dept

2. Outline Background: N-Tuple Classifiers The scanning n-tuple grid Isolated Character Recognition Isolated Face Recognition Convolutional Mode OCR Real time vision demo Conclusions

3. N-Tuple Classifiers Work by randomly sampling input space First applied to binary images Very fast; reasonable accuracy Scanning N-Tuple classifier (Lucas, 1995) Applied to sequence recognition Applied to sequence recognition Fast and accurate Fast and accurate Current work SNT Grid SNT Grid Specially developed for convolutional (sliding window) applications Specially developed for convolutional (sliding window) applications Recognise patterns independent of location Recognise patterns independent of location

4. Likelihood Image SNT-Grid System Architecture Binarise (e.g. Niblack) Scanning Index (SNT-Grid) Integrated Likelihoods Likelihood Image Further Processing (e.g. Dictionary or Language Model)

5. Original

6. Binarised

7. SNT Indexed

8. Simple Operation Slide grid over image Slide grid over image Interpret each position as binary number Interpret each position as binary number

9. Efficient Implementation Very simple idea Decompose one 2-d scan Into two 1-d scans! Reduces time complexity Suppose image is n x n Suppose image is n x n Window is m x m Window is m x m Reduce from O(n 2 m 2 ) Reduce from O(n 2 m 2 ) To O(n 2 ) To O(n 2 ) Well worth the effort!

10. Worked Example

11. SNT Indexing: Java Code

12. OCR Results: MNist Digits

13. SNTGrid Speed on MNist Java Implementation Chars are 28 x 28 grey level images Training (60,000 chars) 8s (> 7,000 cps) 8s (> 7,000 cps) Testing (10,000 chars) 3.8s (> 2,600 cps) 3.8s (> 2,600 cps)

14. ORL Face Data 40 subjects 10 images from each Using 5 for training, 5 for testing Average around 97.5% accuracy Competitive with other methods Much faster!

15. Museum Archive Cards Hard to read with conventional OCR

16. 2 Detector : Raw outputs

17. ‘2’ Detector – Integrated OP (Uses Integral Array of Viola + Jones)

18. Real-time Demo Very efficient Can use it for real-time expression recognition Or a ‘video’ joystick! Bit like EyeToy – but potentially more sophisticated

19. Sample tests Real-time Demo Real-time Demo Real-time Demo

20. Conclusions Basis of simple and efficient computer vision Trick is the scan decomposition Also use of integral image to accumulate likelihoods Currently being applied to reading text in natural scenes Many other applications also Further reading: ICDAR 2005 Paper (on my web page)