1. 1 Querying Graphics through Analysis and Recognition INRIA Lorraine

2. 2 Research fields Image processing and segmentation Structural pattern recognition Statistical pattern recognition Information spotting and retrieval In the context of the analysis and recognition of graphics-rich documents

3. 3 Querying Graphics through Analysis and Recognition

4. 4

5. 5

6. 6

7. 7 Scientific staff Suzanne Collin, Assist. Prof. UHP Philippe Dosch, Assist. Prof. Nancy 2 Bart Lamiroy, Assist. Prof. INPL/Mines Gérald Masini, CR CNRS Salvatore Tabbone, Assist. Prof. U. Nancy 2 Karl Tombre, Prof. INPL/Mines Laurent Wendling, Assist. Prof. UHP PhD students Sabine Barrat, CIFRE contract (pending) Thi Oanh Nguyen, joint supervision with IFI (Hanoi, Vietnam) Oriol Ramos Terrades, joint supervision with UAB, Barcelona (Spain) Jan Rendek, CIFRE France Télécom Jean-Pierre Salmon, FRESH (European project) Zhang Wan, joint supervision with City U. Hong Kong (pending) Daniel Zuwala, MESR grant Technical staff Yamina Smail, Epeires project X, Fresh project (pending) Administrative staff Isabelle Herlich (part time) Françoise Laurent (part time)

8. 8 Main results 2004-05 Hierarchical binarization

9. 9 Focus on symbol recognition – Symbol spotting combining Radon-based signature and structural approach Main results 2004-05

10. 10 Improvement of recognition rates through combination of shape descriptors Main results 2004-05 The set of images I Recognition rates by descriptors

11. 11 Application : extraction of letters in heritage documents DescriptorsCESAGFDTRfWS Before494170595055 After523975695072 Recognition rates DescriptorsCESAGFDTRfWS Before857890898096 After9381979487100 Ranking Improvement of recognition rates through combination of shape descriptors Main results 2004-05

12. 12 Main results 2004-05 Raster-to-vector conversion method based on random sampling and parametric fitting

13. 13 Segmenting the skeleton RANVEC : Random sampling on pairs of vector points Extension of primitive as long as it fits arc or segment (linear regression)

14. 14 Simplification and unification of primitives

15. 15 GREC’01GREC’03GREC’05 WinnerDave EllimanJiQiang SongXavier Hilaire VRI0,6810,6090,803 Arc segmentation contest

16. 16 Application domains/transfer Electrical wiring diagrams in aeronautics  FRESH project (FP6 STREP Aeronautics program)

17. 17 Application domains/transfer Cultural heritage documents  ACI Madonne, FP6 STREP proposal QUIMERA-Doc submitted 9/05

18. 18 QgarLib : library of C++ classes QgarApps : applications QgarGUI : user interface qgar.org, APP Refactoring to professional standards Open architecture (XML) 80,000 lines of C++ code (comments not counted) 30 to 40 downloads of code per month >10 documentation browses per day (robots excluded)

19. 19 Positioning within INRIA Fully within one of INRIA’s 7 challenges in strategic plan: Developing multimedia data and information processing Regular partnership with Imadoc (research group at Irisa) Joint contacts Texmex (Sym-C)/Qgar with industrial partner Recent contacts with Lear on browsing of large image bases

20. 20 Collaborations National: informal consortium Nancy, Rennes, La Rochelle, Rouen, Tours, Lyon with several joint projects (ACI Madonne, RNTL past and submission, Techno-Vision Epeires, IST submission) and coordination of actions CVC/UAB, Barcelona: long lasting relationship, associated team SymbolRec, joint PhD supervisions City University Hong Kong: associated in Epeires, PAI submission accepted, joint PhD supervision IFI, Vietnam: joint PhD supervision University of Auckland (NZ), University of Bern, Carleton University (Canada)

21. 21 Achievements, strengths, weaknesses Leadership position at international level on graphics recognition Announced in project and largely addressed: Symbol recognition and spotting Performance evaluation Strong and adequate applicative backing Improvement in number of PhD students Still low on permanent workforce

22. 22 Future work Scalability of symbol recognition methods Large number of models Variations within the same shape class –Combining structural and statistical methods –Hierarchical approach

23. 23 Future work Complex symbols

24. 24 Future work Dynamic, on-the fly recognition and spotting: from model-based recognition to freehand recognition

25. 25 Future work Multi-modal indexation (text / graphics / image / video) in multimedia and document databases (collaborations with Texmex, Lear, …) Interactivity with user (relevance feedback)

26. 26 Future work Performance evaluation International symbol recognition contests 2003 & 2005 Epeires –French Techno-Vision program –4 universities, FT R&D, 1 company + foreign partners UAB & CityU –www.epeires.orgwww.epeires.org Future research challenges –Simple and non-biased metrics –Ground-truth/recognition output matching methods –Generation of large sets of training and benchmarking data using realistic image degradation models

27. 27 Epeires – ground-truthing

28. 28 Future work Software : increase number of applications