1. - 1/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G DIGIFACT: Digitisation of Museum Artefacts Facilitating Search & Retrieval Eddie Cooke & Robert Keogh C entre for D igital V ideo P rocessing Dublin City University, Ireland

2. - 2/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Outline Background Aims of project Extending 2D catalogueing 3D Image-based rendering 2D/3D Search & Retrieval DigiFact Applications: –Admin Application –Search & Retrieval Application

3. - 3/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Background European Museums: Louvre, Uffizi, Prado, Hermitage, National gallery London National Museum of Ireland (NMI) 4-5 million artefacts, 4 curatorial divisions, 4 separate geographical locations Requirement for collections to be documented so that they are accessible for browsing and searching Catalogueing written descriptions, 2D images which cannot provide concise visual definitions of artefacts 3D models can be created using laser scanners:a laser camera rig with a motion control system No commerically available 3D search system

4. - 4/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Michelangelo Project (Stanford Uni) 2 billion polygons, resolution 0.5mm-50μm Over-kill for the majority of artefacts

5. - 5/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Aims of Project Novel approach to catalogueing museum artefacts supporting low-cost, efficient 3D view generation and combined 2D/3D search and retrieval –Extend existing 2D image cataloguing systems of museums via searchable content-based metadata –Develop a low-cost, intutative method for the creation of 3D views of artefacts –Develop 2D and 3D indexing mechanisms for artefact search and retrieval "Show me this artefact from a different (user defined) angle“, “Find me similar views of similar artefacts"

6. - 6/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Extending 2D catalogueing Currently photographs of the artefacts are stored Provide a system which allows 2D images, captured using our setup, to be automatically indexed initally using the aceToolbox Low-level features from colour, texture and shape

7. - 7/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G 3D Image-Based Rendering (IBR) Geometrical rendering is expensive and time consuming IBR: novel views using a set of 2D reference images along with point correspondences across these images to produce novel views. –Rendering is independent of the artefact's complexity or size and does not require any form of specialised hardware. –Level of realism of the novel view is proportional to the quality of the reference images, which are quickly and easily captured. –Setup is cheap (DigiFact < 2000Euro)

8. - 8/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G 3D Image-Based Rendering (IBR) Digital video camera, motorised turntable

9. - 9/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G 2D/3D Search & Retrieval 2D colour, texture and shape descriptors. Experiments required to identify which are suitable for fusion. No commerically available 3D search & retrieval systems Design descriptors based on: –Disparity Estimation Camera Calibration View Orientation Use virtual views to help identify required view

10. - 10/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G DigiFact Applications 2 Applications –Admin Capture, Create & Process Artefacts –Search and Retrieval Browse Artefacts in Museum Catalogue Search for specific Artefacts

11. - 11/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Admin Application Fig. 1 “Admin” Use Case Model 4 Use Cases –Data Capture –New –Open –Process/Edit

12. - 12/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Rig Setup Fig. 2 Image Capture Rig

13. - 13/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Kaidan Software Fig. 3 Kaidan Software (Capture Screen)

14. - 14/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Data Captured …. ……….. 0 Degrees30 Degrees355 Degrees 5 Degree Intervals = 72 Shots Where do we get 5 Degrees From?

15. - 15/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Admin Application New Artefact Open Artefact Fig. 4 Admin GUI Interactions Data Capture

16. - 16/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Admin Application Edit & Run Segmentation Fig. 5 Editing an Artifact

17. - 17/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Processed Image Data Fig. 6c Dominant Colour XML Fig. 6a Original Image @ 0’ (A) Fig. 6d Image A Segmented Fig. 6e Disparity Map for A with respect to B Fig. 6b Original Image @ 45’ (B)

18. - 18/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G S&R Application Fig. 7 “User” Use Case Model

19. - 19/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G S&R GUI Fig. 8 “User” GUI (Trecvid 2004)

20. - 20/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Virtual View Mug at 0 Degrees Mug at 10 Degrees Mug Warped at 5 Degrees Rectified @ 0 Degrees Rectified @ 10 Degrees

21. - 21/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Technologies Java –GUI, Database Connections C/C++ –Intel’s OpenCV Library (3D info and Image Interpolation) –aceToolbox (2D Descriptors) MatLab –Camera Calibration and Image Rectification MySQL Database

22. - 22/22 - Centre for Digital Video Processing C E N T R E F O R D I G I T A L V I D E O P R O C E S S I N G Thanks for your attention…