2How CryEngine 3 and AMD GCN Architecture Gave Birth to a Red Gem: "Project Phoenix" Bill BilodeauDeveloper Relations EngineerAMDKedhrin GonzalezCreative DirectorIllfonicSean TracySenior Field Applications EngineerCrytekDongsoo HanSoftware Research Engineer
4Creating a new ruby Re-imagining an icon Ruby in the past New design Cinematic FeelEstablished PartnersMaking a game cinematicAuthoring ToolsTools UsedTressFXLeveraging TechnologyCryENGINE 3Mesh Tessellation
5Re-imagining an icon | Ruby In The Past Ruby through the agesShowcasing new technologyAnime Style
6Re-imagining an icon | New Design Modern look and designRealistic “combat ready” RubyBased off realismShowcase materials and tech
7Re-imagining an icon | New Design Showcase Materials and Lighting
8Cinematic Feel | Established Partners The Graphic Film CompanySimon West DirectedExperienced Film TeamEditing, Cinematography, and Animation SupportDigital DomainBody Motion CaptureFacial Motion CaptureArt Bully ProductionsOutsourced Character Models87ElevenStunts and Fight Scenes
9Cinematic Feel | Making A Game Cinematic Make a game first, cinematic secondEnvironment built like a gameFocusing on Next GenHigh pixel density assetsDetailed BackgroundsAssets that can be reusedNew techniques
10Authoring Tools | Tools Used 3D Studio Max, Maya, Mudbox, Crazybump, nDo 2, dDo, Zbrush, Photoshop, Illustrator, Motionbuilder, Shave and a HaircutFull Body Motion CaptureFacial Motion CaptureCryENGINE 3
11Authoring Tools | Tress FX Created hair in Maya using Shave and a HaircutConvert strands to NURB curvesExport with Python scriptImport to Engine and finish settings
13Leveraging Technology | Mesh Tessellation Gain DepthPlanning for PhongPlanning for Displacement
14CryENGINE 3 Features Leveraged For Project Phoenix "Ruby"
15CryENGINE Features Leveraged for ruby Applied Lighting TechnologyReal-Time Area Lights with texture reflectionComposite 3d Lens Flares and FXSpherical and Box Projected Image Based LightingInnovations in Shading and RenderingScreen Space Directional Occlusion – SSDOReal-Time Local Reflections – RLRAnti Aliasing for RubyEye ShaderTessellation and DisplacementDisplacement MappingPhong-During the development of our recently released multiplatform title Crysis 3 there was an immense amount of team work involved across several groups. The improvements made to the CryENGINE are not just on the rendering side: for example the entire engine is now much more multi-core friendly. However, in terms of the Ruby project many of the showcased features are indeed in rendering which is of paramount importance especially for next-generation PC games.- On the Rendering side, which is the main focus of this talk, some of the biggest differences design wise is that we’ve moved into a new hybrid deferred renderer, which helped performance wise with MSAA and modern video cards, particularly in complex scenes.The other big difference is image quality wise, this time we wanted to satisfy every PC aficionado taste/performance requirement: does the user like sharp anti-aliased images? Or does user prefer a bit smoother/blurrier movie like image? So for this engine iteration we introduced a fully fledged DX11 HW multisampling support, with no corners cut on quality as seen is many games, with several different modes.- In Ruby we leverage some older features developed for Crysis 2 like SSDO and RLR, however, some of the newer features that we’ve implemented are specialized area lighting, flares and image based lighting in addition to improved and adaptive tessellation schemes.
16Applied lighting technology Real-Time Area Lights with texture reflectionLight sources in videogames are usually simulated analytically via an infinitely small point light source or directional light source when projection is required.This is a fairly good approximation for most cases, but in the real world light can come from anywhere and has a volume, this affects shadows and light reflection appearance.Many situations in Ruby when light sources cannot be simple points.Billboards, Neon Lamps, Translucent materialsToo many point lights required to simulate these effects!Besides great art usage, the magic behind our high fidelity lighting results is in essence a physically based foundation and an investment in the motto real time all the time. This “real time all the time” approach has been in place since CryENGINE3 development began, with linear correct math, high dynamic range rendering, volume area lights and image based lighting.Real-time Area Lights Real-Time Volume lighting or Area Lights is a recent feature introduced into the CryENGINE 3. Light sources in videogames are usually simulated analytically via an infinitely small point light source or directional light source when projection is required. This is a fairly good approximation for most cases, but in the real world light can come from anywhere and has a volume, this affects shadows and light reflection appearance.For example, your PC monitor casts light from a rectangular shaped volume. We try to approximate such cases with this technique.For the Ruby Project, the real-time area lights are used for artificial lighting, when the actual light sources are so large that their point-specular highlights would become a visual issue. They also provide very smooth shadows, depending on the size of the light sources, in opposition to the standard point light (projectors) that offer very sharp shadows.To give the Ruby project the fidelity it required too many point lights would have needed to be used to achieve it.
17Applied lighting technology Point LightsArea LightsPoint lights cast infinitely Sharp shadows.Real Shadows never look so sharp.CryENGINE uses shadow maps which allows them to be slightly blurred with a fixed width.Area Light is a light that has volume.Its shadow starts sharp near to the object but blurs more and more at distance.Much more physically plausibleWhat are area lights anyways?Point Lights cast infinitely sharp shadows. Doing this with stencil shadows would be achievable but in real life shadows never look this sharp.In CE3 we use Shadow maps which can be blurred. We even use a variable penumbra soft shadow when the sun is used.With Area lights the shadow is sharper near the object but blurs the further the shadow is from the caster. This is far more physically plausible and has been in the past too expensive to do in real time. Until now.
18Applied lighting technology For the Ruby Project, the real-time area lights are used for artificial lighting, when the actual light sources are so large that their point-specular highlights would become a visual issue.Area Light Highlight and Texture ReflectionPoint Light Specular HighlightNote the difference in specular highlight shapeFor the Ruby Project, the real-time area lights are used for artificial lighting, when the actual light sources are so large that their point-specular highlights would become a visual issue.
19Applied lighting technology Texture reflection and shape controlThe Ruby project also contains materials that are quite smooth and glossy as the environment is wet and in the rain.Area Lights are quite noticeable in this situation with the shape control over the specular highlights and with accurate texture reflection.Area Light Texture ReflectionPoint Light Helper ShapeArea Light Helper ShapeThe Ruby project also contains materials that are quite smooth and glossy as the environment is wet and in the rain. Area Lights are quite noticeable in this situation with the shape control over the specular highlights and with accurate texture reflection.Area Light Helper Shape
20Applied lighting technology 3d Composite Lens FlaresProcedural Flares seem great, but in practice we need Artistic Input!A usual solution for such challenges is to hardcode a couple cases and that’s it.We created something much more user friendly and to help each project have its own look without a programmer having to write anything new.In Ruby, most key-lights are able to produce flares, orbs, streaks and other kind of visual aberrations due to the scratches and irregularities on the lens.As a fact, “lens flares” are trendy in this industry at the moment due to their more frequent use in movies and are often over-used or over-exaggerated, which makes them too intrusive and have the tendency to annoy certain type of players.They can have gameplay value though, by making certain part of the scene more appealing and thus helping to lead the player to their objective.
21Applied lighting technology 3d Composite Lens FlaresComposite Flare with 3 Sub EffectsComposite Flare with 1 Sub EffectsComposite Flare with 2 Sub EffectsNo FlaresFirst image shows a composite flare on the headlights using 3 sub effects. Streaks, lens orbs and glowSecond image (below) shows only 2 sub effects. Lens orbs and glowThird image shows only glowLast image shows all effects disabled.
22Applied lighting technology Image Based Lighting – IBLIBL is a rendering technique where complex lighting is stored in a environment map which is projected onto the scene.Positive points:High qualityFast for many lights and even a complex environment is reflectedEnergy preserving specular powerNegative pointsWorks only well for local positions with distant emitters and reflection contentStatic content onlyIn the CryENGINE IBL implementation diffuse lighting can be approximated very well by convolving an environment map (diffuse convolution) which is stored as a cube map again. Because of bilinear filtering, the texture can be quite low resolution as seen here.Hard specular reflections are simple as a single texture lookup in the mirrored eye direction gives the lighting in that directionFor Ruby it was very important to leverage the Image Based Lighting techniques within the CryENGINE. IBL is a rendering technique where complex lighting is stored in a environment map which is projected onto the scene. In simple words, a light probe or environment map is just an image on a sphere projected outwards.In the CryENGINE IBL implementation diffuse lighting can be approximated very well by convolving an environment map (diffuse convolution) which is stored as a cube map again. Because of bilinear filtering, the texture can be quite low resolution. Mip maps are not required and the result with mip maps can actually look worse as ordinary mip mapping on the GPU is computed for each 2x2 pixel block and 2x2 block artifacts can become noticeable.
23Applied lighting technology Depending on the light probe, the lighting can be more ambient or harsh or even colored. The following images show some examples (including sun which casts shadow)Depending on the light probe, the lighting can be more ambient or harsh or even colored.1. Green diffuse, white specular, high glossiness (specular power, affects how polished the material appears) 2. Grey diffuse, gray specular, high glossiness 3. Light gray diffuse, black/no specular (complete Lambert material but note that there are still shades on the part facing away from the sun) 4. Bright gray diffuse, gray specular, medium glossiness (appears more dull) 5. Red diffuse, red specular (even reflections are colored now, this could be even a different color), high glossiness 6. Dark diffuse, bright specular, high glossiness 7. Gray diffuse, dark specular, high glossiness 8. Same as 5 but additionally with a normal map 9. Dark diffuse, dark specular, high glossiness, normal map
24Applied lighting technology Box Projected Image Based Lighting – IBLEssential to the ruby demo was an advancement made for IBL probes called box projected IBL.This is a good technique for flat mirrors.In the Ruby demo this is how we achieving very accurate reflections without scrolling or rotating the IBL probe.Box Projection More AccurateSpherical Projection Less AccurateEssential to the ruby demo was a small advancement made for IBL probes called box projected IBL.This is a good technique for flat mirrors.It operates best in interiors as with outdoor scenery a large bounding box value effectively reduces the technique back to a spherical projection look. In the Ruby demo this is how we achieving very accurate reflections without scrolling or rotating the IBL probe.
25SHADING AND RENDERING Screen Space Directional Occlusion Contact Shadows or Screen Space Directional Occlusion (SSDO) is a novel technique developed for Crysis 2.SSDO is more physically correct than our former technique SSAO.We came up with a new implementation which is efficient enough to allow contact shadows to be applied from every light source in the worldSSDO can also help fixing self-shadowing issues and greatly improves the global lighting quality, especially when many different objects interact with each other from a lighting standpoint.SSDO OFFSSDO ONContact Shadows or Screen Space Directional Occlusion (SSDO) is a novel technique that is used in the CryENGINE3. SSDO is more physically correct than our former technique SSAO.It provides smooth contact shadows that are especially noticeable when using ambient deferred lights which do not cast traditional shadow.We came up with a new implementation which is efficient enough to allow contact shadows to be applied to every light source in the world. It also retains the color information and applies it to the contact shadow. The SSDO can also help fixing self-shadowing issues and greatly improves the global lighting quality, especially when many different objects interact with each other from a lighting standpoint.
26SHADING AND RENDERING Screen Space Directional Occlusion SSDO provides smooth “free” contact shadows that are especially noticeable when using ambient deferred lights which do not cast traditional shadow.It retains the color information and applies it to the contact shadow. Red and Blue Light is seen here bleeding into each other for Purple Contact ShadowsSSDO OffSSDO OnSSDO provides smooth “free” contact shadows that are especially noticeable when using ambient deferred lights which do not cast traditional shadow.It retains the color information and applies it to the contact shadow.
27SHADING AND RENDERING Real time Local reflections Reflections are one of the biggest challenges to solve efficiently in real-time rendering, particularly for deferred rendering/lighting based engines.We introduced a novel approach we call real-time local reflections (RLR).RLR is a screen space approximation to ray-traced HDR reflections, from and on all objects on screen.We use a ray marching technique combined with blur and jitter to disguise low sample counts.Although the results are not always perfect this technique allows for any kind of curved surface in the scene to efficiently reflect the nearby surroundings in real time, including self-reflections which are not possible with cube maps or planar reflections.Reflections are one of the biggest challenges to solve efficiently in real-time rendering, particularly for deferred rendering/lighting based engines.We introduced a novel approach we call real-time local reflections (RLR).RLR approximates ray-traced HDR reflections from and on all objects on screen. Although the results are not always perfect this technique allows for any kind of curved surface in the scene to efficiently reflect the nearby surroundings in real time, including self-reflections which are not possible with cube maps or planar reflections
28SHADING AND RENDERING Real time Local reflections Currently uses 9 ray marched samples but was increased to 32 for Ruby as GCN Hardware manages the high sample count quite well because of efficient flow control hardware.Can be increased further as hardware becomes even more powerful.RLR OffRLR 32 SamplesDepending on the sharpness and fidelity required sample count can be increased as well as adding or removing blur and jittering noise.
29SHADING AND RENDERING Anti Aliasing The latest engine iteration introduces the richest set of Anti aliasing options ever available within the CryENGINE.Anti-Aliasing and its effectiveness can vary from user to user as some prefer a very sharp image, whilst others prefer a blurrier image.CryENGINE gives developers and their players all of the most effective and technically sophisticated modes to suit their preferences. Each mode has very different implications in terms of overall performance, quality and sharpness. The CryENGINE now supports the following anti-aliasing techniques with user controlled settings for number of samples and overall quality per mode.FXAA (Fast-Aproximate Antialiasing)SMAA (Subpixel Morphological Antialiasing)MSAA (the foundation DX11 support for SMAA)And also no AA, since surprisingly some users prefer no AA at all.10 modes with varying degrees of performance vs. Image Quality tradeoffs.
30SHADING AND RENDERING Fast Approximate Anti-Aliasing – FXAA This is the mode with the best performance of the pack, while offering fairly reasonable image quality. It’s a post process solution, hence it doesn’t tackle any sub pixel information, some shimmering might be noticeable on slow camera movements due to lack of sub pixel movement.No AAFXAAFast Approximate Anti-Aliasing – FXAAThis is the mode with the best performance of the pack, while offering fairly reasonable image quality. It’s a post process solution, hence it doesn’t tackle any sub pixel information, some shimmering might be noticeable on slow camera movements due to lack of sub pixel movement.
31SHADING AND RENDERING Improved Eye Shader The new eye shader uses Iris Parallax Mapping by default. The old alpha-blended method has been deprecated as it doesn't work with deferred lighting.Luckily, fixing the assets requires very little work.The new shader was designed with merged textures in mind (character head texture contains eyes and overlay in the same texture).Improved eye shader with Iris Parallax mapping. Even uses Dynamic Pupil adaption depending on the light within a scene or can be controlled via material parameters. Glint and subsurface further increases the believability of the eyes.
32Tessellation Tessellation and Displacement One of the most important updates provided by the DirectX 11 API was the introduction of Programmable Hardware Tessellation.With the improvements to tessellation artists working on Ruby no longer had to go through a process of pre-tessellated their assets before seeing them in engine, thus allowing them to set tessellation in real- time by clicking a checkbox on a per-material basisDisplacement mappingTessellates the mesh uniformly and Reads a displacement height map to extrude the vertices.PhongPhong is an approximation technique designed to smooth, based on vertex normals. The only drawback is that the surface is not perfectly smoothed across patch boundaries and can look a bit inflated.All Tessellations Schemes are now adaptive mitigating the cost on a per patch basis of all tessellated meshes.One of the most important updates provided by the DirectX 11 API was the introduction ofProgrammable Hardware Tessellation. Since our initial implementation in the earlier version of the CryENGINE 3 many changes had to be made to the tessellation pipeline as the workflow was awkward for artists.With the improvements to tessellation artists working on Ruby no longer had to go through a process of pre-tessellated their assets (before seeing them in engine) thus allowing them to set tessellation in real-time at just by clicking a checkbox on a per-material basis
33TessellationRuby’s arm and Hand without Tessellation not some hard polygons edges on the fingers and hands.
34TessellationWith displacement mapping and pn triangles enable the silhouette is smoother and the robotic coils are extruded.
36Realistic hair rendering and simulation Also used in Tomb Raider Ruby Hair | TressFXRealistic hair rendering and simulationAlso used in Tomb RaiderGoes beyond simple shells and fins representation used in gamesHair is rendered as actual strands with self shadowing, antialiasing and transparencyPhysical simulation using GPU compute shadersVery flexible to allow for different hair styles and different conditions
37Ruby Hair Rendering | TressFX What goes into good hair?Anti-aliasingVolumetric self shadowingTransparencyThousands of individual hair strandsTessellation not necessaryBased on I3D 2012 paper from AMD“A Framework for Rendering Complex Scattering Effects on Hair”, Yang, et al.Algorithms modified for improved real-time performanceAnti-Aliasing+ Volume Shadows+ Transparency
38Ruby Hair Rendering | TressFX Kajiya Hair Lighting ModelAnisotropic hair strand lighting modelUses the tangent along the strand instead of the normal for light reflectionsInstead of cos(N, H) , use sin(T,H)Marschner ModelTwo specular highlightsPrimary light colored highlight shifted towards the tipSecondary hair colored highlight shifted towards the root
39Anti-Aliasing Every hair strand is anti-aliased manually Not using Hardware MSAA!Compute pixel coverage on edges of hair strands and convert it to an alpha valueNot using Hardware MSAA!: this means you get anti-aliasing even when HW MSAA or SSAA is not used. Additional reason why real transparency support is essential.
40Ruby Hair Rendering | TressFX Self Shadowing Uses a simplified Deep Shadow Map techniqueNo Self ShadowsWith Self Shadows
41Ruby Hair Rendering | TressFX Transparency Order Independent Transparency (OIT) using a Per-Pixel Linked Lists (PPLL)Fragments are stored in link lists on the GPUNearest K fragments are sorted and used for renderingNo TransparencyWith Transparency
43Ruby Hair Simulation | TressFX Physics Simulation Hair strand is represented as polylines with vertices and edgesLC(Length constraints) is for inextensible hair.Efficient global and local shape constraints to simulate various hair stylesGSC(Global shape constraints) help preserve hair style and initial shapeIt can guarantee that initial hair style can be restored even after strong agitationsLSC(Local shape constraints)For bending/twisting forces to support various hair styles i.e. straight, curly or wavyStable time integration with Verlet methodCollision detection and response between hair and model using capsules
44Ruby Hair Simulation | TressFX Physics Simulation The most difficult part of simulation was to figure out how to simulate stylized hair.In other words, how to hold curly hair shape?To do it, bending and twisting forces are crucial.As in robotics, an open-chain structure has joints and each joint has parent-child relationships to its connected joints.With local transforms in chain structure, we can get a global transformWe find goal positions using local/global transforms for eachvertex and update vertex position by interpolating it usingstiffness value.𝑤 𝑇 𝑖 = 𝑤 𝑇 0 ∙ 0 𝑇 1 …∙ 𝑖−2 𝑇 𝑖−1 ∙ 𝑖−1 𝑇 𝑖
45Ruby Hair Simulation | TressFX Physics Simulation Hair can be assigned to the different groupsAllows different simulation parameters such as stiffness and dampingFine tuning mechanism for usersVarious hair conditions can be simulated such as wet or heavy on the flyA simple aerodynamics for windAll simulation is processed inside GPU using Compute Shader in DX11Mixed hair vertex and strand level parallel processes utilize GPU architecture efficiently.By combining with vertex instancing, the actual simulated number of hair can be lower – guide hairsdrywet
46ReferencesXuan Yu, Jason C. Yang, Justin Hensley, Takahiro Harada, and Jingyi Yu, A Framework for Rendering Complex Scattering Effects on Hair, I3D 2012.Dongsoo Han and Takahiro Harada, Real-time Hair Simulation with Efficient Hair Style Preservation, VRIPHYSJ. Kajiya and T. Kay. Rendering Fur with Three Dimensional Textures, SIGGRAPH 89 Conference Proceedings, pp , 1989Stephen R. Marshner, Henrik Wann Jensen, Mike Cammarano, Steve Worley, and Pat Hanrahan, Light Scattering from Human Hair Fibers, In Proceedings of SIGGRAPH 2003.TressFX rendering implementation by Karl Hillesdale, Research Engineer, AMD.
47Additional References . Hayward, K, "Reflections with Billboard Imposters", with-billboard-impostors.html. Huang, X. "Think DirectX11 Tessellation! Lots of Options", GDC 2010 (link: us/download/details.aspx?id=27397 (batch link together with tittle name)) . BEHC "Box Projected Cubemap Environment Mapping" , environment-mapping/. McDonald, J. "Tessellation on Any Budget", GDC (http://developer.amd.com/wordpress/media/2012/10/Tessellation_On_Any_Budget-GDC2011.pps). Kasyan, N. , Schulz, N., Sousa, T. "Secrets of the CryENGINE 3 Graphics Technology", Siggraph (link:. Lagarde, S. "Local Image Based Lighting With Parallax Corrected Cubemaps", Siggraph 2012 (link:. Sousa, T. , Wenzel, C. , Raine, C. "The Rendering Technologies of Crysis 3", GDC 2013Hungry for more? Check out these references.