Presentation is loading. Please wait.

Presentation is loading. Please wait.

Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,

Similar presentations


Presentation on theme: "Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,"— Presentation transcript:

1 Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center, San Jose

2 SISA CSL “Innovate by doing” VW Technology Trends Analysis 1.State of The Art 1.1 Virtual World Clients for Fixed Platform 1.2 Virtual World Clients for Mobile Platform 1.3 Virtual World Server Technologies 2.Catalyst Elements for VW 2.1 3D content 2.2 Natural interaction for 3D 2.3 Artificial Intelligence in VW 3.Mixed Reality as a Killer Application Backup Slides Mixed Reality Workgroup Mission and Goals Issues in 3D - 1 -

3 SISA CSL “Innovate by doing” - 2 - Trends: 1. Variety of VW clients: - standalone, Web-based (ActiveX or Flash), CE-based (game console), etc 2. Multimodal in-world communication (e.g. textual + bubble + voice chat) 3. Support of user created 3D content via standard format (e.g. COLLADA) Challenges: Insufficient client performance: 3D rendering quality, physics, animation Voice chat is becoming standard, but it still doesn’t benefit from non- verbal communication (i.e., non-intuitive avatar gesture/action control) Complicated 3D content creation pipeline: CAD/CAM skills are required Trends: 1. Variety of VW clients: - standalone, Web-based (ActiveX or Flash), CE-based (game console), etc 2. Multimodal in-world communication (e.g. textual + bubble + voice chat) 3. Support of user created 3D content via standard format (e.g. COLLADA) Challenges: Insufficient client performance: 3D rendering quality, physics, animation Voice chat is becoming standard, but it still doesn’t benefit from non- verbal communication (i.e., non-intuitive avatar gesture/action control) Complicated 3D content creation pipeline: CAD/CAM skills are required 1.1 State of The Art: VW Clients for Fixed Platforms VW Web Plug-In VW on CE Devices Sony’s Playstation Home VIVATY Scene for AIM Opportunities: Actively apply multicore/parallel computing technologies (for both CPU and GPU) on VW client development Sensor technologies that will accurately translate physical motions into virtual actions Design intuitive and intelligent UI for easier creation and/or manipulation of virtual objects Opportunities: Actively apply multicore/parallel computing technologies (for both CPU and GPU) on VW client development Sensor technologies that will accurately translate physical motions into virtual actions Design intuitive and intelligent UI for easier creation and/or manipulation of virtual objects Flash-based VW WebFlock SmallWorlds Standards for 3D content Forterra’s Olive VW Platform supporting COLLADA* Current status: The Electric Sheep Company: Providing Flash-based & private- labeled VW service, WebFlock SMALLWORLDS: Flash-based VW VIVATY: Combining Web plug-in Virtual Community (VIVATY Scene) with social networking services (AIM and Facebook) SONY: Providing VW community service (HOME) based on the game console Playstation3 FORTERRA: Support for COLLADA* into its open and standards-based VW platform, OLIVE Current status: The Electric Sheep Company: Providing Flash-based & private- labeled VW service, WebFlock SMALLWORLDS: Flash-based VW VIVATY: Combining Web plug-in Virtual Community (VIVATY Scene) with social networking services (AIM and Facebook) SONY: Providing VW community service (HOME) based on the game console Playstation3 FORTERRA: Support for COLLADA* into its open and standards-based VW platform, OLIVE

4 SISA CSL “Innovate by doing” Trends: 1. Mobile 3D clients start providing users with permanent access to VWs 2. Connecting VW users in real-time from instant messaging client 3. Emergence of motion sensors and haptic feedback on mobile devices Challenges: Limited performance of remote rendering & streaming technologies No voice/text enabled thin client connecting VW on mobile platform Lack of new use models for sensor-mounted mobile devices Trends: 1. Mobile 3D clients start providing users with permanent access to VWs 2. Connecting VW users in real-time from instant messaging client 3. Emergence of motion sensors and haptic feedback on mobile devices Challenges: Limited performance of remote rendering & streaming technologies No voice/text enabled thin client connecting VW on mobile platform Lack of new use models for sensor-mounted mobile devices 1.2 State of The Art: VW Clients for Mobile Platforms State-of-Art Mobile Device for 3D GraphicsMobile VW client Vollee’s SL client Vivox’s SLim client Opportunities: Enhance the performance of 3D graphic accelerating processor for mobile devices Integrate VoIP/messaging system with mobile immersive 3D client Develop highly-efficient mobile VW client and its UI mechanism Opportunities: Enhance the performance of 3D graphic accelerating processor for mobile devices Integrate VoIP/messaging system with mobile immersive 3D client Develop highly-efficient mobile VW client and its UI mechanism 3D Motion Tracking Apple’s iPhone with ARM1176JZF & Samsung S3C6400 Thin VW client - 3 - Current status: VOLLEE: Leader in mobile VW service with remote rendering & streaming technologies VIVOX (w/ Linden Lab): First commercial voice-driven IM client (SLim) for Second Life APPLE: iPhone is equipped with a top-level chip (ARM1176JZF) with 3D accelerating coprocessor (Samsung S3C6400) Current status: VOLLEE: Leader in mobile VW service with remote rendering & streaming technologies VIVOX (w/ Linden Lab): First commercial voice-driven IM client (SLim) for Second Life APPLE: iPhone is equipped with a top-level chip (ARM1176JZF) with 3D accelerating coprocessor (Samsung S3C6400)

5 SISA CSL “Innovate by doing” Trends: 1. Open-source approach to interoperable server platform 3. Efforts for building MMOG environments in VW 2. Both open and private-labeled virtual world (VW) services are increasing Challenges: Client-neutral, flexible and highly-trusted server platform is required Difficult to attract and hold a large volume of concurrent users Server maintenance cost is still expensive (per user) Trends: 1. Open-source approach to interoperable server platform 3. Efforts for building MMOG environments in VW 2. Both open and private-labeled virtual world (VW) services are increasing Challenges: Client-neutral, flexible and highly-trusted server platform is required Difficult to attract and hold a large volume of concurrent users Server maintenance cost is still expensive (per user) 1.3 State of The Art: VW Server Technologies MMOG* in VW Multiverse Opportunities: Develop software standards for interoperable platforms for online 3D applications and virtual worlds Connect different hardware platforms (e.g. mobile devices and fixed systems) via VW applications Rethink VW platform architecture (e.g. cloud computing) Opportunities: Develop software standards for interoperable platforms for online 3D applications and virtual worlds Connect different hardware platforms (e.g. mobile devices and fixed systems) via VW applications Rethink VW platform architecture (e.g. cloud computing) Interoperable VWOpen-Source Community Open Grid by IBM+Linden Lab OpenSimulator - 4 - Current status: OpenSIM: Open-sourced and flexible 3D application server supporting various VW clients IBM + Linden Lab: Releasing the first interoperable VW service platform (Open Grid Beta) Multiverse: Providing open development toolkit for MMOG- type VW, including authoring tools and source code Current status: OpenSIM: Open-sourced and flexible 3D application server supporting various VW clients IBM + Linden Lab: Releasing the first interoperable VW service platform (Open Grid Beta) Multiverse: Providing open development toolkit for MMOG- type VW, including authoring tools and source code

6 SISA CSL “Innovate by doing” Trends: 1. Renaissance of 3D movies - 11 movie titles released (from 2005 to 2008) and many more in 2009 2. Consistent popularity of games with 3D content 3. Explosive increase in medical imaging with 3D content ($1.4B by 2013) 4. Web-based VW and Mirror World (SL, Google Earth) 5. Stereo Synthesis - 2D to 3D conversion, depth capturing, multi-aperture imaging Challenges: Displays: auto-stereoscopic displays are expensive and cause user fatigue and most of 3D viewing solutions require wearing awkward glasses UX: there is no good solution in the market for natural and intuitive user interface for 3D content 2.1 Catalyst Elements: 3D Content Opportunities: Create innovative and intuitive user interaction model for immersive 3D VW client on consumer electronic devices Provide mobile solutions for 3D content navigation 3D interactive movies for VW Remote medical service and training in VW - 5 - Current status: Philips: Developed auto- stereoscopic displays. Technical issues such as narrow angle of view and user fatigue. No compelling UX 3DV Systems: Detection of body gestures using depth cameras for avatar control. Issues of user fatigue and lack of resolution GestureTek: Control of digital content by tracking gestures. Could lead to user fatigue Source: Insight Media Source: 3D SMPTE standard activities Wendy Aylsworth (Warner Bros.)

7 SISA CSL “Innovate by doing” Trends: 1. Beyond keyboard and mouse: evolution towards alternative HCI tools 2. Multi-touch: Evolution towards gesture interfaces on surface 3. Multitude of sensors: Progressive adoption of orientation, position and motion sensing (optical sensors, inclinometers, accelerometers, compass) Challenges: User fatigue: solutions that rely on mapping human body movements often lead to user fatigue Lack of feedback: it is difficult to give appropriate feedback for 3D interaction Trends: 1. Beyond keyboard and mouse: evolution towards alternative HCI tools 2. Multi-touch: Evolution towards gesture interfaces on surface 3. Multitude of sensors: Progressive adoption of orientation, position and motion sensing (optical sensors, inclinometers, accelerometers, compass) Challenges: User fatigue: solutions that rely on mapping human body movements often lead to user fatigue Lack of feedback: it is difficult to give appropriate feedback for 3D interaction 2.2 Catalyst Elements: Natural UI for 3D Opportunities: Combine different technologies to provide a more immersive and engaging UX for 3D interaction Create closed-loop systems for user experience and 3D object rendering by dynamically displaying 3D object according to user position and interaction with virtual content Opportunities: Combine different technologies to provide a more immersive and engaging UX for 3D interaction Create closed-loop systems for user experience and 3D object rendering by dynamically displaying 3D object according to user position and interaction with virtual content OrangeSeeingmachinesSoftkinetic Mgestyk - 6 - Current status: Seeingmachines: Developer of face API that tracks human face with normal webcam and displays images based on view-dependent- rendering Softkinetic: Provides 3D real-time gesture recognition middleware Mgestyk: Combines 3DV systems depth camera with hand-gesture processing. Current status: Seeingmachines: Developer of face API that tracks human face with normal webcam and displays images based on view-dependent- rendering Softkinetic: Provides 3D real-time gesture recognition middleware Mgestyk: Combines 3DV systems depth camera with hand-gesture processing.

8 SISA CSL “Innovate by doing” Trends: 1. A.I. engines programmed to learn by interacting with humans in VW - VW gives advantage of embodiment to A.I. engine without hassle or costs that comes with physical robotics - VW allows exponential increase in interaction for AI engines to learn 2. There are efforts to make VW presence and interaction more engaging Challenges: Recreate self-mind in VW: there is need to create a self-awareness as present in human interaction by appropriately planning Pattern mining: finding complex patterns requires prohibitively inefficient searches Self-modeling: creating learning algorithms capable to embody intelligence Language and gesture processing: interpretation of language, gestures Trends: 1. A.I. engines programmed to learn by interacting with humans in VW - VW gives advantage of embodiment to A.I. engine without hassle or costs that comes with physical robotics - VW allows exponential increase in interaction for AI engines to learn 2. There are efforts to make VW presence and interaction more engaging Challenges: Recreate self-mind in VW: there is need to create a self-awareness as present in human interaction by appropriately planning Pattern mining: finding complex patterns requires prohibitively inefficient searches Self-modeling: creating learning algorithms capable to embody intelligence Language and gesture processing: interpretation of language, gestures 2.3 Catalyst Elements: Artificial Intelligence in VW Opportunities: Virtual pets can be a killer application for A.I. in VWs Adding ability to have pets genuinely learn and respond to the environment will make them more real to the user and increase the user/virtual pet bond Autonomous avatars for creating rich virtual environments Opportunities: Virtual pets can be a killer application for A.I. in VWs Adding ability to have pets genuinely learn and respond to the environment will make them more real to the user and increase the user/virtual pet bond Autonomous avatars for creating rich virtual environments - 7 - Current status: Novamente IRC: methodology for teaching: Imitative learning: instructor shows by example Reinforcement learning:– student repeats movements with feedback Corrective learning:– instructor actively corrects student’s movements Current status: Novamente IRC: methodology for teaching: Imitative learning: instructor shows by example Reinforcement learning:– student repeats movements with feedback Corrective learning:– instructor actively corrects student’s movements

9 SISA CSL “Innovate by doing” Trends: 1. Increase in information overlay (Googlemaps, VWs) 2. See-through devices as a platform for mixing VW and RW (iPhone App) Challenges: Design of wearable displays: current solutions are bulky and too geeky Need for an intelligent agent: user should not be overwhelmed by profusion of virtual data but system should have an embedded agent that filters relevant or preferred information Lack of precision of location sensors: current location sensors need to improve precision in order to identify location for data augmentation Trends: 1. Increase in information overlay (Googlemaps, VWs) 2. See-through devices as a platform for mixing VW and RW (iPhone App) Challenges: Design of wearable displays: current solutions are bulky and too geeky Need for an intelligent agent: user should not be overwhelmed by profusion of virtual data but system should have an embedded agent that filters relevant or preferred information Lack of precision of location sensors: current location sensors need to improve precision in order to identify location for data augmentation 3. Mixed Reality as a Killer Application Opportunities: Develop technologies for an unobtrusive system with natural UX for mixed reality Possible applications: -Travel planning with shared experience and enhanced participation -Training of first responders -Inter-reality physics: virtual reality system coupled with real-world one Opportunities: Develop technologies for an unobtrusive system with natural UX for mixed reality Possible applications: -Travel planning with shared experience and enhanced participation -Training of first responders -Inter-reality physics: virtual reality system coupled with real-world one Virtual Continuum (by Paul Milgram, 1994) - 8 - Current status: BBC R&D: Have explored the impact of mixed reality in media: http://www.bbc.co.uk/rd/projects/virtu al/mixtv/index.shtml http://www.bbc.co.uk/rd/projects/virtu al/mixtv/index.shtml Nokia Research: Research on mobile mixed reality system with augmentation of real world using mobile phones. GaTech: Augmented Reality in SL Current status: BBC R&D: Have explored the impact of mixed reality in media: http://www.bbc.co.uk/rd/projects/virtu al/mixtv/index.shtml http://www.bbc.co.uk/rd/projects/virtu al/mixtv/index.shtml Nokia Research: Research on mobile mixed reality system with augmentation of real world using mobile phones. GaTech: Augmented Reality in SL Nokia Research BBC R&D

10 SISA CSL “Innovate by doing” Backup slides

11 SISA CSL “Innovate by doing” Mixed Reality Workgroup: Mission We believe that Mixed Realities have the potential to change the way we interact with the world in a radical way. Under Mixed Reality, we refer to the merging of real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time. But radical change can only happen if the technology is available on a broad basis. The more people who can interact with Mixed Realities, the clearer the advantages of such a technology will become. Therefore, our mission is to improve human interaction with real and virtual worlds by encouraging the mass adoption of Mixed Reality technologies, and the creation of an industry with very low barriers to innovation and business success.

12 SISA CSL “Innovate by doing” Mixed Reality Workgroup: Goals We propose to provide information aimed at technology providers, investors, and business decision makers: Casebook that documents past efforts, lessons learned, and best practices Roadmap of key technical requirements for "pacing mass usages“ “Grand Challenges” collection to inspire innovation in the industry We are an Active Workgroup that develops documents which frame the usages, as well as key technical and business model barriers, to the success of the Mixed Reality vision.

13 SISA CSL “Innovate by doing” Issues in 3D 3D content creation - Quality x Quantity - Mastering 3D content for home x cinema - 3D creation and workflow training and education 3D content storage, transmission and distribution - File formats, compression, transmission - Coordinate work in standard bodies (DCI, MPEG, IEEE 3D Consortium, 3D4YOU, etc…) 3D Promotion - Common naming conventions and vocabulary - Develop content for web distribution - End-user and retailer/reseller education and in-store demos 3D Displays - 3D display quality metrics – ghosting, brightness, etc… - 3D human factors – eye fatigue, viewing time, viewing angle dependency, etc

14 SISA CSL “Innovate by doing” 3D Display Technologies

15 SISA CSL “Innovate by doing” 3DTV vs. 3D-ready TV

16 SISA CSL “Innovate by doing” Cost

17 SISA CSL “Innovate by doing” PDP display market forecast Samsung manufactures 3D-ready PDP-TV. This is the expected PDP forecast in the USA with 3D TVs dominating the market in the next years.

18 SISA CSL “Innovate by doing” 3D LCD display market forecast It is expected that 3D LCD is going to emerge in the market and grow exponentially

19 SISA CSL “Innovate by doing” 3D and 3D-ready display market

20 SISA CSL “Innovate by doing” 3D content – movie as a driving force

21 SISA CSL “Innovate by doing” TV – How do you know it is 3D-ready?

22 SISA CSL “Innovate by doing” 3D-ready TV – 3D glasses and PC requirements

23 SISA CSL “Innovate by doing” 3D-ready TV – 3D glasses and PC requirements

24 SISA CSL “Innovate by doing” 3D-ready TV – PC requirements


Download ppt "Virtual Worlds Technology Roadmap December 2008 Seung Wook Kim, Francisco Imai, Stefan Marti HCI Research Team, Computer Science Lab Samsung R&D Center,"

Similar presentations


Ads by Google