1. Image-based Techniques for the Creation and Display of Photorealistic Interactive Virtual Environments George Drettakis 1, Maria Roussou 3,4, Nicolas Tsingos 1, Alex Reche 1,2, Emmanuel Gallo 1,2 1 REVES/INRIA, Sophia-Antipolis, France 2 CSTB, France 3 University College London, UK 4 Foundation of the Hellenic World, Hellas (Greece)

2. overall goals n Realistic VE’s –simplify model creation –enhance realism of display –interactivity n Education and design –enhance learning process –help design and decision making

3. previous approaches n Limited level of realism in VE‘s –creation of environments slow, costly and with limited realism –display of environments with low realism n missing shadows and vegetation, flat textures n missing sound n no life (crowds, traffice etc.)

4. previous approaches n Many developments in Computer Graphics, but rarely transferred to VE’s –considered too expensive in resources –engineering aspect overwhelming n Examples of CG techniques –image-based modeling [Debevec 96/98] –point-based rendering [Stamminger 01] –perspective shadow maps [Stamminger 02]

5. overview of our approach n Project context –CREATE IST project n Three components –creating and displaying a realistic VE –interactivity –user-centered design and evaluation

6. model creation n Image-based modeling –shoot panoramas, stitch and render cube maps

7. resulting model n After calibration, modelling & texturing

8. video n Creation of model using Stitcher and ImageModeler (improved versions)

9. data acquisition for CREATE n Two applications areas –cultural heritage and education –urban planning and design n Two corresponding sites –Messene site in Greece (archeology) –Place Garibaldi in Nice (tramway project)

10. view-dependent 3D models n One common model, multiple textures –use blending for view-dependent display n For complex scenes use layering –perform geometric sort to isolate image layers –use a view-dependent texture per layer

11. view-dependent 3D models

12. urban planning models synthetic image

13. enhanced virtual environment

15. video n Place Garibaldi simulation –shadows –sound –point-based rendering

16. interactivity n Cultural heritage –manipulation of temple elements –reconstruction of temple n Urban planning –design and construct “urban features” such as benches or tents

17. user-centered design and evaluation n User task analysis –expert and novice users n Cultural heritage –experts: archeologists –novice users: general public, children n Urban planning –experts: architects –novice users: general public, decision makers

18. user task analysis n Interviews with users, analysis of existing task flow –augment the current tools –add realism and interactivity to enhance productivity n Preliminary studies –cultural heritage with children

19. preliminary formative studies n Children in a resizing experiment –7 to 12 years old –CAVE environment n Architects for urban planning –simple observation –description of needs

20. current work n Completion of enhanced VE tools –more realistic trees –integration of crowds and sound n Real-task experiments –architects in a design task for urban planning (under way)

21. video n Interactivity –design and placement of benches and tents

22. conclusions n Image based techniques –improve realism of VE’s –combined with interactivity and user- centered design show promise for improved usage of VE’s

23. acknowledgements n CREATE is a 3-year RTD project funded by the 5th Framework Information Society Technologies (IST) Programme of the EuropeanUnion (IST-2001-34231), http://www.cs.ucl.ac.uk/create/. The partners of CREATE are University College London (UK), Foundation of the Hellenic World (GR), INRIA (FR), REALVIZ (FR), University of Cyprus (CY), PERCRO (I) and CSTB (FR) http://www.cs.ucl.ac.uk/create/ n Alias|Wavefront for their donation of Maya n Alexandre Olivier-Mangon for all the image-modeling and virtual modeling, and the videos n Alex Reche and Emmanuel Gallo are supported by CIFRE doctoral fellowships INRIA-CSTB

24. thank you http://www-sop.inria.fr/reves http://www.fhw.gr http://www.cstb.fr http://www.cs.ucl.ac.uk/create