Presentation on theme: "Digital Interactive Entertainment Dr. Yangsheng Wang Professor of Institute of Automation Chinese Academy of Sciences"— Presentation transcript:
Digital Interactive Entertainment Dr. Yangsheng Wang Professor of Institute of Automation Chinese Academy of Sciences
Overview of Digital Entertainment Industry
Digital Entertainment Industry Year 2008 the market size will be about $100B and major products are TV games 、 PC games and mobile games. Microsoft, Sony and Nintendo provide most digital entertainment products. China has a very big potential market for digital entertainment products.
Internet and Game in China The total amount of Chinese Internet users is up to 87 millions in 2004, about 25% are online game(OG) users. OGs are one of major parts of digital entertainment contents in China. Almost half of the OGs on the domestic market mainly come from Korea. Some Chinese game companiers are beginning to develop their own OGs. PC Stand-alone games are losing their market. Mobile games are developed rapidly, but the income is on the low level at the present stage. Overseas game companies enter Chinese market via Joint-venture.
source ： CGPA&IDC 2004 Internet Users in China
Online Game Market in China source ： CGPA&IDC 2004
Stand-alone Game Market in China source ： CGPA&IDC 2004
New Research & Applications of Natural Interactive Technologies
Traditional games mainly depend on mouse, keyboard, joystick and special input device. Game players desire more natural and intelligent interaction methods, including voice, gesture, expression, gaze, and intelligent sensor, etc. User-oriented multi-model HCI (Human Computer Interaction) technologies has become the hotspot of research and application in many domains. Natural Interactive Technologies (NIT)
Computer Vision for Computer Games and Graphics Camera is used as a powerful interface device for games by sensing user’s body position, head orientation, direction of gaze, pointing commands, and gestures. Developer: Mitsubishi Electric Research Laboratories. Applications: Sega games NIT Research and Application-I Vision device
Vision-based Human-Computer Interface for Games By a camera, the player's body movements are projected right into the game as an unencumbered input and control device. Developer: Sony Computer Entertainment America Applications: Sony PlayStation2 EyeToy games NIT Research and Application-II Camera
Voice Control for Games and Simulations Many of today's games have a lot of keyboard commands that can be hard to remember. Speech recognition technologies are used in this area for game control. Product: Game Commander (Mindmaker) Product: Game Voice (Microsoft SideWinder) NIT Research and Application-III
Interactive Entertainment Research Group in CASIA
Auto-Construction of a Game Object; Player-Oriented Game Multi-model Human & Computer Interaction; Natural Interactive Entertainment Platform. The Outline of Our Research
Photo-stereo Based Model Reconstruction Images capture Normal calculation Surface reconstruction flashlight camera bracket control card
Images capture Images of the surface under different lighting conditions is captured in a very short period, that the register is avoid.
Normal calculation A photo-stereo method is used to retrieve the surface normal based on the reflect property of the surface.
Surface reconstruction The relevant height between the points can be integrated from the derivative field. A pyramid integration method is used to optimize the results
Orthogonal images Based Model Reconstruction Generic Model and its feature points selection Two images feature points editing Generic model deformation Texture merging and mapping
Generic model and its feature points selection 79 feature points according MPEG4
Images feature points editing and texture merging frontal faceside face texture merging result
Generic model deformation and texture mapping texture generic model model reconstruction result deforming
Individual game characters Individual faces give players more fun and immersion. player A player B
Face detection Deformation & Texture mapping Expression recognition Expression retargeting & Animation Video-Based Model Reconstruction and Animation Capture user’s frontal face with a camera, then deform an existent face model and map the face image to create an individual face model. Five kind of user’s expressions are recognized real-time to drive the relevant animations.
Auto-Extraction of key points of a face An RealBoost-based statistical learning is adopted to build a model for each facial point. During extraction, the models are used to locate facial points for each face image.
Expression recognition and retargeting Expression recognition Source expression models Individual expression results surpriseanger sad happyfear A novel approach to real time facial expression recognition using an embedded HMM which uses observation vectors that are composed of DCT results.
Getting into a Game - Change face Player’s frontal face is captured by a camera. Player’s face feature points are detected to deform character’s face model. Player’s face image is used change character’s face texture.
Head detection and pose estimation Fist detection and motion analysis Other motion detection and analysis Video-Based Motion Detection and Analysis
Head detection and pose estimation frontleftrightupdownclosefar locating the position of a head recognizing the pose of the head The novel scheme based on Markov model fusing appearance, color, and motion information.
Fist detection and motion analysis prototype DEMO training samples
Driving a car using your hands and voice Player’s hands control the car’s direction. Player’s voice controls the car’s speed and status.
Motion Detection and Analysis Project video into game Motion detection Parameters calculation Game objects control
Playing a game with your body Player’s body movement is projected right into a game The motion detection and analysis results are used to activate objects in a game.
Online-Game Strategic Research Group
A meeting of the group The group includes major digital entertainment companies, universities, research institutes, and digital entertainment association in China.
Write strategic report of digital entertainment for Chinese government. Establish “China National Center for Digital Entertainment”. The goal of the Strategic Research Group