How CryEngine 3 and AMD GCN Architecture Gave Birth to a Red Gem: "Project Phoenix" Bill Bilodeau Developer Relations Engineer AMD Kedhrin Gonzalez Creative Director Illfonic Sean Tracy Senior Field Applications Engineer Crytek Dongsoo Han Software Research Engineer

Creating A New Ruby

Creating a new ruby Re-imagining an icon Ruby in the past New design Cinematic Feel Established Partners Making a game cinematic Authoring Tools Tools Used TressFX Leveraging Technology CryENGINE 3 Mesh Tessellation

Re-imagining an icon | Ruby In The Past Ruby through the ages Showcasing new technology Anime Style

Re-imagining an icon | New Design Modern look and design Realistic “combat ready” Ruby Based off realism Showcase materials and tech

Re-imagining an icon | New Design Showcase Materials and Lighting

Cinematic Feel | Established Partners The Graphic Film Company Simon West Directed Experienced Film Team Editing, Cinematography, and Animation Support Digital Domain Body Motion Capture Facial Motion Capture Art Bully Productions Outsourced Character Models 87Eleven Stunts and Fight Scenes

Cinematic Feel | Making A Game Cinematic Make a game first, cinematic second Environment built like a game Focusing on Next Gen High pixel density assets Detailed Backgrounds Assets that can be reused New techniques

Authoring Tools | Tools Used 3D Studio Max, Maya, Mudbox, Crazybump, nDo 2, dDo, Zbrush, Photoshop, Illustrator, Motionbuilder, Shave and a Haircut Full Body Motion Capture Facial Motion Capture CryENGINE 3

Authoring Tools | Tress FX Created hair in Maya using Shave and a Haircut Convert strands to NURB curves Export with Python script Import to Engine and finish settings

Leveraging Technology | CryENGINE 3 Complete Toolset Lighting Animation FX Materials Rapid Development Deployment from authoring tools Flexible Pipeline

Leveraging Technology | Mesh Tessellation Gain Depth Planning for Phong Planning for Displacement

CryENGINE 3 Features Leveraged For Project Phoenix "Ruby"

CryENGINE Features Leveraged for ruby Applied Lighting Technology Real-Time Area Lights with texture reflection Composite 3d Lens Flares and FX Spherical and Box Projected Image Based Lighting Innovations in Shading and Rendering Screen Space Directional Occlusion – SSDO Real-Time Local Reflections – RLR Anti Aliasing for Ruby Eye Shader Tessellation and Displacement Displacement Mapping Phong -During the development of our recently released multiplatform title Crysis 3 there was an immense amount of team work involved across several groups. The improvements made to the CryENGINE are not just on the rendering side: for example the entire engine is now much more multi-core friendly. However, in terms of the Ruby project many of the showcased features are indeed in rendering which is of paramount importance especially for next-generation PC games. - On the Rendering side, which is the main focus of this talk, some of the biggest differences design wise is that we’ve moved into a new hybrid deferred renderer, which helped performance wise with MSAA and modern video cards, particularly in complex scenes. The other big difference is image quality wise, this time we wanted to satisfy every PC aficionado taste/performance requirement: does the user like sharp anti-aliased images? Or does user prefer a bit smoother/blurrier movie like image? So for this engine iteration we introduced a fully fledged DX11 HW multisampling support, with no corners cut on quality as seen is many games, with several different modes. - In Ruby we leverage some older features developed for Crysis 2 like SSDO and RLR, however, some of the newer features that we’ve implemented are specialized area lighting, flares and image based lighting in addition to improved and adaptive tessellation schemes.

Applied lighting technology Real-Time Area Lights with texture reflection Light sources in videogames are usually simulated analytically via an infinitely small point light source or directional light source when projection is required. This is a fairly good approximation for most cases, but in the real world light can come from anywhere and has a volume, this affects shadows and light reflection appearance. Many situations in Ruby when light sources cannot be simple points. Billboards, Neon Lamps, Translucent materials Too many point lights required to simulate these effects! Besides great art usage, the magic behind our high fidelity lighting results is in essence a physically based foundation and an investment in the motto real time all the time. This “real time all the time” approach has been in place since CryENGINE3 development began, with linear correct math, high dynamic range rendering, volume area lights and image based lighting. Real-time Area Lights   Real-Time Volume lighting or Area Lights is a recent feature introduced into the CryENGINE 3. Light sources in videogames are usually simulated analytically via an infinitely small point light source or directional light source when projection is required. This is a fairly good approximation for most cases, but in the real world light can come from anywhere and has a volume, this affects shadows and light reflection appearance. For example, your PC monitor casts light from a rectangular shaped volume. We try to approximate such cases with this technique. For the Ruby Project, the real-time area lights are used for artificial lighting, when the actual light sources are so large that their point-specular highlights would become a visual issue. They also provide very smooth shadows, depending on the size of the light sources, in opposition to the standard point light (projectors) that offer very sharp shadows. To give the Ruby project the fidelity it required too many point lights would have needed to be used to achieve it.

Applied lighting technology Point Lights Area Lights Point lights cast infinitely Sharp shadows. Real Shadows never look so sharp. CryENGINE uses shadow maps which allows them to be slightly blurred with a fixed width. Area Light is a light that has volume. Its shadow starts sharp near to the object but blurs more and more at distance. Much more physically plausible What are area lights anyways? Point Lights cast infinitely sharp shadows. Doing this with stencil shadows would be achievable but in real life shadows never look this sharp. In CE3 we use Shadow maps which can be blurred. We even use a variable penumbra soft shadow when the sun is used. With Area lights the shadow is sharper near the object but blurs the further the shadow is from the caster. This is far more physically plausible and has been in the past too expensive to do in real time. Until now.

Applied lighting technology For the Ruby Project, the real-time area lights are used for artificial lighting, when the actual light sources are so large that their point-specular highlights would become a visual issue. Area Light Highlight and Texture Reflection Point Light Specular Highlight Note the difference in specular highlight shape For the Ruby Project, the real-time area lights are used for artificial lighting, when the actual light sources are so large that their point-specular highlights would become a visual issue.

Applied lighting technology Texture reflection and shape control The Ruby project also contains materials that are quite smooth and glossy as the environment is wet and in the rain. Area Lights are quite noticeable in this situation with the shape control over the specular highlights and with accurate texture reflection. Area Light Texture Reflection Point Light Helper Shape Area Light Helper Shape The Ruby project also contains materials that are quite smooth and glossy as the environment is wet and in the rain. Area Lights are quite noticeable in this situation with the shape control over the specular highlights and with accurate texture reflection. Area Light Helper Shape

Applied lighting technology 3d Composite Lens Flares Procedural Flares seem great, but in practice we need Artistic Input! A usual solution for such challenges is to hardcode a couple cases and that’s it. We created something much more user friendly and to help each project have its own look without a programmer having to write anything new. In Ruby, most key-lights are able to produce flares, orbs, streaks and other kind of visual aberrations due to the scratches and irregularities on the lens. As a fact, “lens flares” are trendy in this industry at the moment due to their more frequent use in movies and are often over-used or over-exaggerated, which makes them too intrusive and have the tendency to annoy certain type of players. They can have gameplay value though, by making certain part of the scene more appealing and thus helping to lead the player to their objective.

Applied lighting technology 3d Composite Lens Flares Composite Flare with 3 Sub Effects Composite Flare with 1 Sub Effects Composite Flare with 2 Sub Effects No Flares First image shows a composite flare on the headlights using 3 sub effects. Streaks, lens orbs and glow Second image (below) shows only 2 sub effects. Lens orbs and glow Third image shows only glow Last image shows all effects disabled.

Applied lighting technology Image Based Lighting – IBL IBL is a rendering technique where complex lighting is stored in a environment map which is projected onto the scene. Positive points: High quality Fast for many lights and even a complex environment is reflected Energy preserving specular power Negative points Works only well for local positions with distant emitters and reflection content Static content only In the CryENGINE IBL implementation diffuse lighting can be approximated very well by convolving an environment map (diffuse convolution) which is stored as a cube map again. Because of bilinear filtering, the texture can be quite low resolution as seen here. Hard specular reflections are simple as a single texture lookup in the mirrored eye direction gives the lighting in that direction For Ruby it was very important to leverage the Image Based Lighting techniques within the CryENGINE. IBL is a rendering technique where complex lighting is stored in a environment map which is projected onto the scene. In simple words, a light probe or environment map is just an image on a sphere projected outwards. In the CryENGINE IBL implementation diffuse lighting can be approximated very well by convolving an environment map (diffuse convolution) which is stored as a cube map again. Because of bilinear filtering, the texture can be quite low resolution. Mip maps are not required and the result with mip maps can actually look worse as ordinary mip mapping on the GPU is computed for each 2x2 pixel block and 2x2 block artifacts can become noticeable.

Applied lighting technology Depending on the light probe, the lighting can be more ambient or harsh or even colored. The following images show some examples (including sun which casts shadow) Depending on the light probe, the lighting can be more ambient or harsh or even colored. 1. Green diffuse, white specular, high glossiness (specular power, affects how polished the material appears) 2. Grey diffuse, gray specular, high glossiness 3. Light gray diffuse, black/no specular (complete Lambert material but note that there are still shades on the part facing away from the sun) 4. Bright gray diffuse, gray specular, medium glossiness (appears more dull) 5. Red diffuse, red specular (even reflections are colored now, this could be even a different color), high glossiness 6. Dark diffuse, bright specular, high glossiness 7. Gray diffuse, dark specular, high glossiness 8. Same as 5 but additionally with a normal map 9. Dark diffuse, dark specular, high glossiness, normal map

Applied lighting technology Box Projected Image Based Lighting – IBL Essential to the ruby demo was an advancement made for IBL probes called box projected IBL. This is a good technique for flat mirrors. In the Ruby demo this is how we achieving very accurate reflections without scrolling or rotating the IBL probe. Box Projection More Accurate Spherical Projection Less Accurate Essential to the ruby demo was a small advancement made for IBL probes called box projected IBL. This is a good technique for flat mirrors. It operates best in interiors as with outdoor scenery a large bounding box value effectively reduces the technique back to a spherical projection look. In the Ruby demo this is how we achieving very accurate reflections without scrolling or rotating the IBL probe.

SHADING AND RENDERING Screen Space Directional Occlusion Contact Shadows or Screen Space Directional Occlusion (SSDO) is a novel technique developed for Crysis 2. SSDO is more physically correct than our former technique SSAO. We came up with a new implementation which is efficient enough to allow contact shadows to be applied from every light source in the world SSDO can also help fixing self-shadowing issues and greatly improves the global lighting quality, especially when many different objects interact with each other from a lighting standpoint. SSDO OFF SSDO ON Contact Shadows or Screen Space Directional Occlusion (SSDO) is a novel technique that is used in the CryENGINE3. SSDO is more physically correct than our former technique SSAO. It provides smooth contact shadows that are especially noticeable when using ambient deferred lights which do not cast traditional shadow. We came up with a new implementation which is efficient enough to allow contact shadows to be applied to every light source in the world. It also retains the color information and applies it to the contact shadow.  The SSDO can also help fixing self-shadowing issues and greatly improves the global lighting quality, especially when many different objects interact with each other from a lighting standpoint.

SHADING AND RENDERING Screen Space Directional Occlusion SSDO provides smooth “free” contact shadows that are especially noticeable when using ambient deferred lights which do not cast traditional shadow. It retains the color information and applies it to the contact shadow.  Red and Blue Light is seen here bleeding into each other for Purple Contact Shadows SSDO Off SSDO On SSDO provides smooth “free” contact shadows that are especially noticeable when using ambient deferred lights which do not cast traditional shadow. It retains the color information and applies it to the contact shadow. 

SHADING AND RENDERING Real time Local reflections Reflections are one of the biggest challenges to solve efficiently in real-time rendering, particularly for deferred rendering/lighting based engines. We introduced a novel approach we call real-time local reflections (RLR). RLR is a screen space approximation to ray-traced HDR reflections, from and on all objects on screen. We use a ray marching technique combined with blur and jitter to disguise low sample counts. Although the results are not always perfect this technique allows for any kind of curved surface in the scene to efficiently reflect the nearby surroundings in real time, including self-reflections which are not possible with cube maps or planar reflections. Reflections are one of the biggest challenges to solve efficiently in real-time rendering, particularly for deferred rendering/lighting based engines. We introduced a novel approach we call real-time local reflections (RLR). RLR approximates ray-traced HDR reflections from and on all objects on screen.  Although the results are not always perfect this technique allows for any kind of curved surface in the scene to efficiently reflect the nearby surroundings in real time, including self-reflections which are not possible with cube maps or planar reflections

SHADING AND RENDERING Real time Local reflections Currently uses 9 ray marched samples but was increased to 32 for Ruby as GCN Hardware manages the high sample count quite well because of efficient flow control hardware. Can be increased further as hardware becomes even more powerful. RLR Off RLR On @ 32 Samples Depending on the sharpness and fidelity required sample count can be increased as well as adding or removing blur and jittering noise.

SHADING AND RENDERING Anti Aliasing The latest engine iteration introduces the richest set of Anti aliasing options ever available within the CryENGINE. Anti-Aliasing and its effectiveness can vary from user to user as some prefer a very sharp image, whilst others prefer a blurrier image. CryENGINE gives developers and their players all of the most effective and technically sophisticated modes to suit their preferences. Each mode has very different implications in terms of overall performance, quality and sharpness.  The CryENGINE now supports the following anti-aliasing techniques with user controlled settings for number of samples and overall quality per mode. FXAA (Fast-Aproximate Antialiasing) SMAA (Subpixel Morphological Antialiasing) MSAA (the foundation DX11 support for SMAA) And also no AA, since surprisingly some users prefer no AA at all. 10 modes with varying degrees of performance vs. Image Quality tradeoffs.

SHADING AND RENDERING Fast Approximate Anti-Aliasing – FXAA This is the mode with the best performance of the pack, while offering fairly reasonable image quality. It’s a post process solution, hence it doesn’t tackle any sub pixel information, some shimmering might be noticeable on slow camera movements due to lack of sub pixel movement. No AA FXAA Fast Approximate Anti-Aliasing – FXAA This is the mode with the best performance of the pack, while offering fairly reasonable image quality. It’s a post process solution, hence it doesn’t tackle any sub pixel information, some shimmering might be noticeable on slow camera movements due to lack of sub pixel movement.

SHADING AND RENDERING Improved Eye Shader The new eye shader uses Iris Parallax Mapping by default. The old alpha-blended method has been deprecated as it doesn't work with deferred lighting. Luckily, fixing the assets requires very little work. The new shader was designed with merged textures in mind (character head texture contains eyes and overlay in the same texture). Improved eye shader with Iris Parallax mapping. Even uses Dynamic Pupil adaption depending on the light within a scene or can be controlled via material parameters. Glint and subsurface further increases the believability of the eyes.

Tessellation Tessellation and Displacement One of the most important updates provided by the DirectX 11 API was the introduction of Programmable Hardware Tessellation. With the improvements to tessellation artists working on Ruby no longer had to go through a process of pre-tessellated their assets before seeing them in engine, thus allowing them to set tessellation in real- time by clicking a checkbox on a per-material basis Displacement mapping Tessellates the mesh uniformly and Reads a displacement height map to extrude the vertices. Phong Phong is an approximation technique designed to smooth, based on vertex normals. The only drawback is that the surface is not perfectly smoothed across patch boundaries and can look a bit inflated. All Tessellations Schemes are now adaptive mitigating the cost on a per patch basis of all tessellated meshes. One of the most important updates provided by the DirectX 11 API was the introduction of Programmable Hardware Tessellation. Since our initial implementation in the earlier version of the CryENGINE 3 many changes had to be made to the tessellation pipeline as the workflow was awkward for artists. With the improvements to tessellation artists working on Ruby no longer had to go through a process of pre-tessellated their assets (before seeing them in engine) thus allowing them to set tessellation in real-time at just by clicking a checkbox on a per-material basis

Tessellation Ruby’s arm and Hand without Tessellation not some hard polygons edges on the fingers and hands.

Tessellation With displacement mapping and pn triangles enable the silhouette is smoother and the robotic coils are extruded.

TressFX Hair Technology in the Ruby Demo

Realistic hair rendering and simulation Also used in Tomb Raider Ruby Hair | TressFX Realistic hair rendering and simulation Also used in Tomb Raider Goes beyond simple shells and fins representation used in games Hair is rendered as actual strands with self shadowing, antialiasing and transparency Physical simulation using GPU compute shaders Very flexible to allow for different hair styles and different conditions

Ruby Hair Rendering | TressFX What goes into good hair? Anti-aliasing Volumetric self shadowing Transparency Thousands of individual hair strands Tessellation not necessary Based on I3D 2012 paper from AMD “A Framework for Rendering Complex Scattering Effects on Hair”, Yang, et al. Algorithms modified for improved real-time performance Anti-Aliasing + Volume Shadows + Transparency

Ruby Hair Rendering | TressFX Kajiya Hair Lighting Model Anisotropic hair strand lighting model Uses the tangent along the strand instead of the normal for light reflections Instead of cos(N, H) , use sin(T,H) Marschner Model Two specular highlights Primary light colored highlight shifted towards the tip Secondary hair colored highlight shifted towards the root

Anti-Aliasing Every hair strand is anti-aliased manually Not using Hardware MSAA! Compute pixel coverage on edges of hair strands and convert it to an alpha value Not using Hardware MSAA!: this means you get anti-aliasing even when HW MSAA or SSAA is not used. Additional reason why real transparency support is essential.

Ruby Hair Rendering | TressFX Self Shadowing Uses a simplified Deep Shadow Map technique No Self Shadows With Self Shadows

Ruby Hair Rendering | TressFX Transparency Order Independent Transparency (OIT) using a Per-Pixel Linked Lists (PPLL) Fragments are stored in link lists on the GPU Nearest K fragments are sorted and used for rendering No Transparency With Transparency

TressFX Hair Simulation in the Ruby Demo

Ruby Hair Simulation | TressFX Physics Simulation Hair strand is represented as polylines with vertices and edges LC(Length constraints) is for inextensible hair. Efficient global and local shape constraints to simulate various hair styles GSC(Global shape constraints) help preserve hair style and initial shape It can guarantee that initial hair style can be restored even after strong agitations LSC(Local shape constraints) For bending/twisting forces to support various hair styles i.e. straight, curly or wavy Stable time integration with Verlet method Collision detection and response between hair and model using capsules

Ruby Hair Simulation | TressFX Physics Simulation The most difficult part of simulation was to figure out how to simulate stylized hair. In other words, how to hold curly hair shape? To do it, bending and twisting forces are crucial. As in robotics, an open-chain structure has joints and each joint has parent-child relationships to its connected joints. With local transforms in chain structure, we can get a global transform We find goal positions using local/global transforms for each vertex and update vertex position by interpolating it using stiffness value. 𝑤 𝑇 𝑖 = 𝑤 𝑇 0 ∙ 0 𝑇 1 …∙ 𝑖−2 𝑇 𝑖−1 ∙ 𝑖−1 𝑇 𝑖

Ruby Hair Simulation | TressFX Physics Simulation Hair can be assigned to the different groups Allows different simulation parameters such as stiffness and damping Fine tuning mechanism for users Various hair conditions can be simulated such as wet or heavy on the fly A simple aerodynamics for wind All simulation is processed inside GPU using Compute Shader in DX11 Mixed hair vertex and strand level parallel processes utilize GPU architecture efficiently. By combining with vertex instancing, the actual simulated number of hair can be lower – guide hairs dry wet

References Xuan Yu, Jason C. Yang, Justin Hensley, Takahiro Harada, and Jingyi Yu, A Framework for Rendering Complex Scattering Effects on Hair, I3D 2012. Dongsoo Han and Takahiro Harada, Real-time Hair Simulation with Efficient Hair Style Preservation, VRIPHYS 2012 . J. Kajiya and T. Kay. Rendering Fur with Three Dimensional Textures, SIGGRAPH 89 Conference Proceedings, pp 271-280, 1989 Stephen R. Marshner, Henrik Wann Jensen, Mike Cammarano, Steve Worley, and Pat Hanrahan, Light Scattering from Human Hair Fibers, In Proceedings of SIGGRAPH 2003. TressFX rendering implementation by Karl Hillesdale, Research Engineer, AMD.

Additional References . Hayward, K, "Reflections with Billboard Imposters", http://graphicsrunner.blogspot.de/2008/04/reflections- with-billboard-impostors.html . Huang, X. "Think DirectX11 Tessellation! Lots of Options", GDC 2010 (link: http://www.microsoft.com/en- us/download/details.aspx?id=27397 (batch link together with tittle name)) . BEHC "Box Projected Cubemap Environment Mapping" , http://www.gamedev.net/topic/568829-box-projected-cubemap- environment-mapping/ . McDonald, J. "Tessellation on Any Budget", GDC 2011 (http://developer.amd.com/wordpress/media/2012/10/Tessellation_On_Any_Budget-GDC2011.pps) . Kasyan, N. , Schulz, N., Sousa, T. "Secrets of the CryENGINE 3 Graphics Technology", Siggraph 2011 (link: http://www.crytek.com/download/S2011_SecretsCryENGINE3Tech.ppt) . Lagarde, S. "Local Image Based Lighting With Parallax Corrected Cubemaps", Siggraph 2012 (link: http://seblagarde.wordpress.com/2012/11/28/siggraph-2012-talk/) . Sousa, T. , Wenzel, C. , Raine, C. "The Rendering Technologies of Crysis 3", GDC 2013 Hungry for more? Check out these references.

bill.bilodeau@amd.com seanp@crytek.com kedhrin@illfonic.com Questions? For follow-up questions, email: bill.bilodeau@amd.com seanp@crytek.com kedhrin@illfonic.com

Disclaimer & Attribution The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. There is no obligation to update or otherwise correct or revise this information. However, we reserve the right to revise this information and to make changes from time to time to the content hereof without obligation to notify any person of such revisions or changes. NO REPRESENTATIONS OR WARRANTIES ARE MADE WITH RESPECT TO THE CONTENTS HEREOF AND NO RESPONSIBILITY IS ASSUMED FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. ALL IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE ARE EXPRESSLY DISCLAIMED. IN NO EVENT WILL ANY LIABILITY TO ANY PERSON BE INCURRED FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. AMD, the AMD arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. All other names used in this presentation are for informational purposes only and may be trademarks of their respective owners. [For AMD-speakers only] © 2013 Advanced Micro Devices, Inc. [For non-AMD speakers only] The contents of this presentation were provided by individual(s) and/or company listed on the title page. The information and opinions presented in this presentation may not represent AMD’s positions, strategies or opinions. Unless explicitly stated, AMD is not responsible for the content herein and no endorsements are implied. IllFonic® and the IllFonic® logo are trademarks and/or registered trademarks of IllFonic, LLC throughout the world.

Trademark Attribution AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdictions. Other names used in this presentation are for identification purposes only and may be trademarks of their respective owners. ©2013 Advanced Micro Devices, Inc. All rights reserved.