Presentation on theme: "The Future of Game Engines"— Presentation transcript:
1 The Future of Game Engines Towards Real-time Photo-realistic Rendering and Natural Character AnimationXiaomao WuR&D Software Engineer, CrytekShanghai, Aug. 1st, 2010Good morning, Ladies and Gentalman. I am honored to be here with you today at ChinaGDC in Shanghai. I’m Xiaomao from Crytek. The topic I’m going to deal with is “The future of game engines”, with focus on rendering and animation.
2 Overview Current technologies in game engines Emerging technologies and the futureFirst I will give an overview of the current technologies in game engines, and then the emerging technologies.
3 Current Technologies in Game Engines Let’s start with the first part.
4 Global Illunimation (GI) CryENGINE® 3: Light propagation volume (Dynamic scene + dynamic lights)LPV-based GI in CryENGINE® 3 was proposed by Anton Kaplanyan (Crytek) in 2009Unreal 3 Lightmass (static lighting and effects)Middleware solutions are also available:GeomericsBeast (used by Unity engine)LightsprintGlobal illumination （全局光照）is one of the most interesting topics.In CryEngine3, we have a GI solution called Light Propagation Volume (光线传播体）, which was proposed by Anton last year. The GI in CryEngine3 supports dynamic scene and dynamic lights, which is very challenging.Unreal has a GI solver called Lightmass, which supports static lighting and effects.Middleware solutions are also available, such as Geomerics, Beast and Lightsprint. Unity engine uses Beast for lightmapping and GI.[Geomerics is being integrated into UE3, CCP’s Dust game will use UE+Geomerics solution]
5 PBRT (Photon Mapping), 45 min CryENGINE® 3, 78MentalRay®, 42 minPBRT (Photon Mapping), 45 minCryENGINE® 3, 60We compared the GI performance in CE3 with mentalRay and Photon mapping. The speedup is tremendous and with the same rendering quality.
6 LightsMark by Stepan Hrbek (Now called LightSprint) Real-time GI in CryENGINE®3VideoLightsMark by Stepan Hrbek(Now called LightSprint)Real-timeTested with World of PadmanProvides test binaryLet me show you the GI demo video in CE3.LightsMark now becomes LightSprint. You can download a test binary from their website and give it a try. It’s a quite interesting middleware.Copyright by LightSprint® and Stepan Hrbek
7 SSAO Widely used in video games First proposed by Vladimir Kajalin at Crytek (2006)Used for the first time in Crysis® (2007)Very popular in games: Crysis®, Crysis Warhead™, Gears of War 2, Uncharted 2, Halo, Starcraft II …SSAO (Screen-space ambient occlusion) is widely used in video games since 2007.It was proposed by Vladimir, the principle R&D engineer at Crytek in 2006, and it was first utilized in Crysis published in 2007.Nowadays it’s quite popular in video games such as ……
8 SSAO Without SSAO With SSAO Pros: High quality No pre-processing, no loading time allocationsMemory efficientIndependent on scene cCons:Only visible occluders on the screen are considered May produce artifactsomplexityWork with dynamic scenesWith SSAO, it’s possible to get high-quality rendering in real-time, without pre-processing and no loading-time allocation. It’s also memory efficient and scene independent.However, one of the cons of SSAO is that only visible occluders are considered, which may cause artifacts.These two images show the effect of SSAO. As you can you, without SSAO, many areas such as the crack area between the pipes.Without SSAOWith SSAO
9 Deferred renderingProposed by Michael Deering and his colleagues at SIGGRAPH ’88Decouple geometric scene complexity from shaderComputation and memory bandwidth reduced to those visible portionsMore complex to handle transparencyDepth peeling for order-independent transparency -> additional batches and g-buffer sizeBut DX10, 11 can perform batches fast, no less effective than forward shadingDeferred rendering is another technology that is widely used in game engines.It was proposed by Michael and his colleagues at SIGGRAPH 1988.The basic idea is to decouple geometric scene complexity from shader, by using an intermediate buffer such as Gbuffer.One drawback of deferred rendering is the difficulty to handle transparency. Depth peeling can be used to achieve order-independent transparency in deferred rendering, but at the cost of additional batches and g-buffer size.Fortunately, DX10/11 is capable of performing batches fast enough to maintain interactive frame rates. When order-independent transparency is desired, deferred shading is no less effective than forward shading using the same technique.
10 Deferred rendering Image rendered by CryENGINE® This is an illustration of deferred rendering combined with SSAO in CryEngine.Linear z value of GBufferSSAO from GBuffer
11 Shadows Dynamic Occlusion Maps Shadow maps with screen-space randomized look-upShadow maps with light-space randomized look-upShadow mask texturePhotoCryENGINE ®Read Finding Next Gen-CryEngine2.pdf
12 ShadowsShadow maps for directional light sources (Cascaded shadow maps)Deferred shadow mask generationUnwrapped shadow maps for point lightsVariance shadow maps (for terrain)Sun + 2 point lightsRefer to “Finding Next Gen-CryEngine2.pdf”Object SHObject SH + Terrain VSM
13 Anti-aliasing Edge AA in CryENGINE® Morphological anti-aliasing (God of War III)
16 Copyright by Sony Entertainment God of War IIICopyright by Sony Entertainment
17 Volumetric environment FogSun ShaftCopyright by Moro et al. A Fast Rendering Method for Shafts of Light in Outdoor Scene.Terrain
18 Post Effect Depth of field (with bokeh) Motion blur Lens flares Bloom/glowColor grading
19 DOF with bokeh Motion blur Lens flare Bloom/glow By Sebastian @CryMod Copyright byLens flareByBloom/glowCopyright by
20 Color Grading Change the color style of a game in real-time User provide a reference chartThe engine change the game scene with the color chartUseful to change the game style, and to create expressive tones
26 Integrated with physics Rag-dollHit-reactionSmooth transition between physics and animation data
27 Additive Animations Video Additive animations are relative animations: add to the base animationUsage:BreathingLooking aroundFlinchingPosture changeAnd more…Video
28 Animation Behavior Behavior tree Script-based AI logic Animation Graph A table that defines a character’s behaviorScript-based AI logicLua script defines the behavior of an NPCAnimation GraphA flow graph which defines the behavior
33 DX 11 and opengl 4.0 New features Chances Risk DirectCompute TessellationImproved multi-threadingShader Model 5.0ChancesReal-time high-quality subdivision surfacesReal-time micro-polygon renderingConvenient dynamic shader bindingRiskPipeline change (code side and art side)
34 New Rendering Approach -Point Based Rendering ProsNaturally fits to automatic LOD schemesFreeform modelingAnti AliasingConsMakes sense only on super high detailed scenesEditing tools still rely greatly on trianglesGPUs focus on rasterization
35 New Rendering Approach -Real-time ray-tracing ProsEasy ParallelizeableNaturally leads to high quality shadows and lightingRendering pipeline simplificationAnti AliasingConsStill too slow for realtime applicationGPUs focus on rasterization
36 New Rendering Approach -Real-time ray-tracing Stephan Reiter, Real–time Ray Tracing of Dynamic Scenes, June, 2008That thesis is interesting, complex scene with interactive speed without heavy optimization and recent HWCopyright by Johannes Kepler Universitat and Stephan Reiter
37 New Rendering Approach -Real-time ray-tracing PantaRay (Used for Avatar)Nvidia ported Weta's PantaRay engine to a CUDA-based GPU version25 times faster by utilizing an Nvidia Tesla S GPU-based serverOptix by NvidiaCopyright by Nvidia®, captured from the publically available Optix SDK
38 Server-side rendering 4G networks have a good ground for thatLow ping – a strong requirement for real-time gamesWill be widely deployed in 5-7 yearsCompression of synthesized videoTemporally decompose the video detailsUse perception-based importanceSalience maps + user-side eye-trackingNeed to amortize cloud-rendering cost per user:Linear trendAmortized trendCostNumber of users
39 Heterogeneous system OpenCL SPAP (Same language for all Not as transparent as SPAPSPAP (Same language for allprocessors, GPU+CPU)Research productCopyright by Q. Hou, K, Zhou and B. GuoCopyright by the KHRONOS Group
40 Heterogeneous system Intel MIC: 22nm, 50+ Cores Code Knights Corner Copyright by Intel®
41 General and accurate motion control Geostatistics (Kriging)Optimizing both kernel parameters and weights: better fittingMore accurate interpolation than RBF, Inverse weighting, k-nearest neighbors ….About half speed as RBF: optimization tricksOptimal control theoryProbability motion controlRecent machine learning R&D are driving this direction
42 Interactive & Dynamic Ragdoll Make it more realisticCryPhysics and Nvidia Physx both support itChallenges: real-time performance, stability
43 Natural and Real-time IK Combine with biomechanics infoLearning prio from large motion databasesExtend 12-DOF analytic IKUniversaIK?HumanIK, MonsterIK, AlienIK, AnimalIK ….Learning from artist-defined sparse posesUse data from biometrics study
44 More Automated Rigging System Possible solutionsRegion-of-Influence (J. P. Lewis et al. 2007)Heat-transfer (Baran and Popvic, 2000)Skinning by examplePure automatic skinning won’t available in 2-3 years, but an highly automated method that can decrease manual weight adjustment to a large extent, is possible
45 Robust and Real-time Motion Retargeting Different skeleton topologyAllow manual interaction in game editorDirections:Decompose and re-combineAnalyze the behaviors of different creaturesThrough an intermediate skeletonRely on a fast, robust full-body IK solution to maintain constrainsA simplified workflow of the SIGGRAPH ’08 paper [Chris Hecker et al.]
46 Smarter Characters Perception (speech, facial expression, environment) Social intelligencePredict the actions of others by their motives and emotional states. Game theory, decision theory, emotion modelingLearning-based: learning game players’ behavior and apply to the AI characters: online and offlineKeep updating AI characters’ skills and knowledge at run-time and make them human-likeNeural networkReinforcement learningProducts: DrivatarTM(Gaming research team, Microsoft Research Cambridge)
47 Thank you! Qestions? Crytek is hiring: www.crytek.com/career/ For any questions contact me:Special thanks toCevat Yerli (CEO, Co-funder of Crytek)Carl Jones (Global Director of BD - CryENGINE)Ury Zhilinsky (R&D Manager)Anton Kaplanyan (Research Lead)Ivo Herzeg (Lead R&D animation Engineer)Scott Fitzgerald (Technical Designer)Vaclav Kyba (R&D Engineer)The whole R&D team at Crytek
48 Three reference slides from Cevat & Anton‘s HPG ’10 talk
49 How to design for the future? FactsFixed Resolution for Gaming till 2012HD 1920 x 60 fpsStereoscopic 3D experience: 30 fps per eyeLimited by current consoles hardwareRisk of “Uncanny Valley“ for contentsPerception-driven approaches!Till 2012 majority of games must use artistic style, physics and AI to differentiate!What‘s the current artistic style? Desaturate colors?
50 Challenges of Future Production challenges Technology challenges Switching to a scaleable codebaseThink of parallelism & async jobsMultithreading, schedulingLarger codebases, multiple platforms & APIsProduction challengesCost of assets increase by ~50% annuallyContent, quality increases. More & more „interactive“Think to improve Tools, Pipelines & Bottlenecks to counter-effect , automate Source Back-Ends Resource CompilersThe better the tools, the cheaper and/or the better your output
51 Opportunities in Future Short-term user impact opportunities till 2012The delta in visual opportunities is limited, BUT...for the next 3 Years: Huge gains are possible in Physics, AI and Simulation of Special EffectsFocus around that knowledge can lead to very different designsMid-erm creative opportunitiesFuture console generationsNew Rendering Methods will become availableThe renaissance of graphics will arriveAllows new visual development directions that will rival full CGI feature films qualtyAction point: Link yourself to console cycle