Presentation on theme: "Physically Based Shading"— Presentation transcript:

A simplified artist interface Niklas Hansson Head Teacher Game programming The Game Assembly.

What is it ? Why should I use it ? How do I integrate it into a project ? Lightning base on the physical properties of light. Replacement of Ad Hoc solutions Instead of having Spec Intensity,Spec Gloss,Reflectivity,Fresnel strength map and even a separate reflection gloss factor. With a physically based solution we will be able to collapse this down to two simple maps. Instead of Ad hoc solutions competing we have one stable system.

Boundary : When light goes from one material to another
Basic BRDF Theory Boundary : When light goes from one material to another Light splits up into two directions : reflection and refraction. The amount of light reflected depends on the materials reflective index. Been asked to keep this not to technical Basics are important. Boundrary L is split into ri and t Split Refractive index BRDF Works no matter the materials We will only look at air to other material Image from “Real-Time Rendering 3rd Edition”

Basic BRDF Theory Basic Material - Part is Reflected
- Part enters the material Metal Part is Reflected The part entering the Material is absorbed Image from “Real-Time Rendering 3rd Edition”

Basic BRDF Theory Non metals
- Light that enters the material scatters internally - Is absorbed - and often exits the surface again. Image from “Real-Time Rendering 3rd Edition”

Sub surface scattering
Basic BRDF Theory Sub surface scattering Distance between enter and exit positions are determined by the material. If red pixel and green exit pos needs SSS BRDF cant cope. If other way Works Image from “Real-Time Rendering 3rd Edition”

Basic BRDF Theory Point model
By ignoring distance between exit and enter points we can model lighting with an BRDF. Holds holds true no matter the source of the light. Works for perfectly flat boundaries Most real world materials (except mirrors) doesn’t share this property. Image from “Real-Time Rendering 3rd Edition”

Basic Microfacet BRDF Theory
Microfacets A surface consists of many small perfectly flat surfaces. How much they differ in normal determines how light interact Microfactes Normal differences encoded as a normal distribution The difference from the Phong model to the Blinn Phong model Blinn Phong instead used the microfacet If parallel light hits small solid angle with all light, the rougher the larger solid angle the light will hit. We use a normal distribution function to model this The data it uses is the roughness of the surface (or flattnes would be a better name) Image from “Real-Time Rendering 3rd Edition”

Basic Microfacet BRDF Theory
As you can see the rougher the surface the blurrier reflection and larger specular highlights. Microfacets model sub normal map resolution shapes Images from “Real-Time Rendering 3rd Edition”

Shadowing & Masking Not all micro facets facing half vectors direction contribute to lightning Shadowing Masking Strength of being physically based Values have real value Can update formulas without changing the value Can be used to uppdate difuse to for exampel to Oren-Nayar Rough material not gonna be as reflective due to ridges blocking light Microfacet BRDF cant handle bounces on ridges Images from “Real-Time Rendering 3rd Edition”

So why is specular so important ?
Images from “Everything is Shiny” by John Hable

So why is specular so important ?
Images from “Everything is Shiny” by John Hable

The Microfacet BRDF Microfacet BRDF LEGO parts
Most different BRDF models only contains changes to the D and G term. Except denominator //Is a correction factor that accounts for quantities being transformed between the local space of the micro facets and the overall macro surface The ‘4’ is related to transforming from the space of half-angle directions to other spaces, and the two clamped dot products are foreshortening factors that relate to the way that the visible area of microfacets change in relation to their orientation.

Returns a Value from 0-1 in RGB
Fresnel Reflectance Returns a Value from 0-1 in RGB Fraction of light reflected from optically flat surface given Light directon l and Half vector h Frist part Fresnel reflectivity Gives the reflctivity of the surface vs the viweing and light angle. Returns spectral values Reflected color doesn’t absorb light However they reflect different amounts of lights at different wavelengths We simplify this using RGB just as in the rest of CG

Fresnel Reflectance Depends on refraction index (in our terms the substance of the object) and the light angle. Fresnel curve Light bounce amount varies with angle. Try putting your head next to a door and se a clear reflection appear. Starts on index of refraction which we will call substance Objects with RGB spec has 3 index of refraction Barely changes in beginning. Turns quickly upwards at the end Some materials dip We wont model this Image from “Real-Time Rendering 3rd Edition”

Fresnel Images from “Everything has Fresnel” by John Hable
Some real world examples Images from “Everything has Fresnel” by John Hable

Schlick’s Approximation
Decently Accurate, uses F(0) the refractive index as input. Which is our substance term.

Gives a scalar value between 0 and 1
Geometric term Gives a scalar value between 0 and 1 Contains the fraction of non obscured microfacets Various functions exists most using roughness as input Using the geometric term to cancel out the denominator is extremely cheap as you can just remove booth from the calculation but will result in a to dark specular. Look at lazarovs keynote for good solutions to this.

Distribution function
Calculates factor of microfacets reflecting light towards the eye The function determines size and shape of the specular highlight Distribution function calculates how many microfacets reflects light based on the roughnees Decides specular size and strength The blinn-phong distribution function is widely used in the games industry

Blinn Phong Distribution Functions
Modified Blinn Phong Basic Blinn Phong Doesn’t account for amount of light hitting the surface Using a max cap is not enough Calcualte amount of light hittign surface And multiply with that Cant reflect more light than hits it.

Beckman Distribution function
Conversion : Cook torrence used beckaman More accurate More expensive. Can convert between

Blinn Phong Energy Conversation
If the specular highlight is larger the specular intensity must be lower. N = roughness A surface cant emit more light than hits it Therefore of light is reflected over a larger solid angle. It needs to be weaker We use a normalization factor to adapt light strength to scale. This makes the equation energy conserving The normalization factors differs depending on the distribution N= roughness phong has a very different normalization factor than blinn Phong. This normalization was derived by Fabian Giesen

Blinn Phong Energy Conversation
Formula from physically based shading 2010 Seems vey off ? Pi is lacking Comes from hemisphere integration Applies constantly over all light so can be removed in our case. Just change light scale with pi

Blinn Phong Energy Conversation
Adding in the pi seems to give a correct curve so lets zoom in to make certain

Blinn Phong Energy Conversation
Zooming in it seems that this curve is off with quite a bit in the lower 100 range at least.

Blinn Phong Energy Conversation
Another common approximation as an 8 instead of a 2 at the top. This one fits much more well. An important part is that it fits better at lower values which contains most of the real world materials. So we get some extra precision where it counts. The factor of 8 is also used in the real time rendering book.

Blinn Phong Energy Conversation
This is another common approximation taken from tri Arcs papers but as you can see it plots vs the n+2/8 curve and not the real one

Blinn Phong Energy Conversation
This is another common approximation taken from tri Arcs papers but as you can see it plots vs the n+2/8 curve and not the real one

Blinn Phong Energy Conversation
Mathematically actually this formula is a better match but as you can see we have a higher margin of error on the lower values where most natural objects resides which is why the formula with an 8 is more often used.

Our Normalized equation for specular light
Energy Conversation Our Normalized equation for specular light Lambert Cdiff*PI <=1 Full BRDF Cdiff+Cspec <=1 Our equation Lambert equation for diffuse Using Cdiff for albedo To be energy conserving Cdiff*pi<1 Putting Cdiff and Cspec together the rule is that no material can reflect more light than hits it, This means albedo plus frefractive index need to stay below 1

Ambient Light is just Light
Has a specular component too reflections roughness controls reflections blurriness substance controls reflection strength. Pre blurred cubemaps Ambient is light Has specular Is handled as reflection No need for extra data as we can already caluclate reflected light If not using ansitropic look up you can preblur mip map levels and use roughness as a curve. For spec we use a logaritmic curve to use as much a sthe color space as possible in the range the artist works For Mip map select we use the default roughnees to keep more mip maps in the just slightely blured part.

ATI cubemap gen has nice functions for preblurring mipmaps.
Ambient Light ATI cubemap gen has nice functions for preblurring mipmaps. For correctnes the blurring needs to match your specular light Prefiltered cubemap will start start looking as diffuse light Cube map gen can prefilter mip maps. Doesn’t do it correctly for this Can roll your own or just download and modify

Blurred cubemap Use the angular extent and always sample from top mip.
Use the glossiness for that mipmap level and evaluate max size of highlight for size of angular extent. Use the phong function to evaluate a contribution for all the pixels inside the extent. Cubemap gen uses angular extents with different functions. We want is filtered in the same way as phong (cant use blinn phong as it’s anisotropic) but phong is perfectly round Calculate angular extent per mip using roughness Use phong to evaluate contribution for sample. Normalize Phong simplifies to cosine filtering . Already in Cubemap gen raised to the roughness Phing normalisation can be ignore as its the same for all samples with the same roughnees Doing this makes specular highlight size and reflection bluriness match.

Why physically based ? It’s more realistic thanks to being physically based? Materials does not require as much retweaking under changing lighting conditions. You can create more easily varied materials with lesser number of values It saves artist time and simplifies workflow The top one here isn’t one that I worry about. It might be to some people but I am a pragmatic. Materials that doesn’t need retweaking as much when lighting changes is a good thing and saves time but by itself it is not a seller. The more varied material interface and the easy to create different materials with a low number of values is a big seller especially combined with deferred rendering The last one here however is the seller for me. Saving artists time is just pure financial sense and the reason we call this a simplified artist interface and that’s the issue I with start with.

Simplified Artist interface
Reduces number of maps from 4-5 to 2. Simplifies Junior artist training No longer take albedo and desaturate for intensity And then lower color for gloss We have all seen it Reduces maps vs ad hocc Reduces work amount Simplifies artist mental model A lot of erroneus tutorials around just using filters. Some tutorials talk about taking the inverse of albedo instead. Makes spec depend on albedo brightness not material Brigher gets brighter or darker gets darker, Gloss not really understood just lower color Reflection just for special objects Fresnel no clue Will look like above.

Simplified Artist interface
PBL simplifies this by creating a material interface that makes sense. Substance Is the index of refraction for the material No need to hand paint. Mathematically matches old intensity A interface you can understand. Using existing physical values Using a look up table and masks to select areas and add material Makes substance quick to create Doesn’t need hand painting Matches old intensity

Color picker Easy visual look up Handles specular color properly
Linear and gamma depending on what the project uses Non metals taken from physics books. Metals index for different wavelengths taken from scientific data Feed into a program by sebastian lagarde that integrates them for RGB color space. Here is the swatch we use at The game assembly. We have booth linear and gamm For average materials default values at the bottom Last values to be tweaked. Else you might tweak others values for an incorrect here and never get it right.

Simplified Artist interface
Roughness The old gloss But thanks to Normalization a lot more useful Where hand painting happens Controls reflection blurrinees and spec Surface Roughness that normal maps can’t show Touch Observe Just old gloss with normalization Can make brighter specular thanks to normalisation Makes ideal for hand painting Controls reflection blurrines as well Artists doesn’t need to worry He only needs to think about the roughness of the surface on the level that won’t shop up in the normal maps Using touch to get a feel Looking for fresnel effects at glancing angles.

Example Material rusty Paint
Substance just a mask for material Roughness contain details

Flexibillity Don’t forget albedo and normals importance
We made a test chart with some different materials to test it out and se how different looks we can achieve.

Sample Oskar stålberg This is all rendered in our maya plugin for PBL and is a good example of how flexible material looks you can get with this system. Look especially at the mush rooms and metal vs the main body to the left And the fuse and plastic cables to the right

Substance almost no detail except for big material change
Roughness contains the work Tried on junior and more senior artists Senior directly grasped and improved speed. Juniors took a while to make the shift then increased speed

Lower numbers of values to tweak Smaller tighter gbuffer
Why physically based ? Lower numbers of values to tweak Smaller tighter gbuffer Low bandwitdh Small ALU increase Ideal for deferred rendering While still allowing very varied materials And now the technical reasons Having fewer variables reduces bandwidth Especially when using deffered rendering PBL works for almost all materials right of the bat so you need very few special cass- We get a tighter gbuffer for just a small ALU increasel

How to use it What did we do ? Microfacet BRDF with
Normalized Blinn Phong Fresnel reflectance Void Geometry Function G(h) = 4(n dot l)(n dot v) Prefiltered glossy reflections from cubemap probes Ambient from ambient probes

What we did Cref is the value from the reflection probe
Camb is the value from the ambient probe Fschlick(v,h) is the fresnel term using the view direction instead of the light direction Removed the pi division to just change the light intensity scale Cref is the reflected light in the cubemap Camb is the ambient light for the point. fresnel term for the reflection uses v and h instead of l and h. energy conversation between reflection and ambient. while not 100% correct it does a good job of mimicking real life behavior Other are using a special case reflection fresnel to reduce edge shine. This worked well for us.

Ambient vs reflection Energy Conversation
Energy conversation created better metalic surfaces While still allowing artists to use an albedo texture. Effect is subtle Right side is more metallic Left side seems washed out.

Also solved issues where artist tried to tweak substance and roughness incorrectly
To remove to strong fresnel effect.

Substance and Roughness vs Intensity and Gloss
Lessons learned Substance and Roughness vs Intensity and Gloss Spend time training and communicating to your artists Help them use the new interface Tools is essential Beware of bugs You got to have artist buy in. Need a mind shift. New names helped. We created a maya plugin to allow the artists to preview models. And had some issues due to bugs in it 

Maya plugin Tool was double edges sword. Artist used new interface
Maya visuals differed from game visuals

Reflection and ambient is a big part of PBL
In game they are dependant on light conditions so that their strength match light strength. Using in engine probes. In maya the artist could select maps and tweak light to values that could not exist together in game. Which meeant what they saw in game and engine differed. A solution would be to try and normalzie cubemap vs light strength but getting 100% is imposible. We are looking at using premade light rigs to solve the problem Using an in engine viewer is even better but artists doesn’t like the extra step of leaving maya.

Flexible Kind of projects Space shooter Realtime Strategy
What we did Flexible Kind of projects Space shooter Realtime Strategy First Person shooter Used the techinique in 12 projects Over 4 teams And 3 genres

Space shooter Space shooter was perfect case
Ambient and reflection from skybox. Only thing we would change is use ambient vs reflection energy conversion

RTS games showed up the pipeline problems with maya vs ingame
And to strong fresnel due to lack of energy conversation caused a lot of issues.

Fps project Goals Techniques
Resolve all issues found during the last project Get an easy to light solution that took little rebuild time and would work with just a few spot lights. Techniques All enviroment and ambient probes generated by the ingame engine to achieve consistency. Box projected cubemaps was used for reflections and a variation was used for ambient Simple Multi bounce solution for lightning

Box projectedion made cubemap lookup ansitropic
would require importance sampling and multipel filtering to look weel. Solution was to interpolate between box proejcted look up and ordinary In engine generated ambient and reflection resolved earlier issues

Specular and reflection doing more work
Occlusion Specular and reflection doing more work Will look weird in areas normally don’t reach by much light Use Ambient occlusion Multiply to spec Multiply to reflection If needed add special Specular occlusion Since booth specular and reflection is doing a lot more work We get problems Dark crevices shines up due to reflection. Made worse by the fact that reflection is added to normal color makes reflection in dark creavices overly shiny. A solution is using ambeint occlusion Holds the amount of direction that light can hit the surface from. The more hid the darker. Will keep your crevice dark. With everything as geometry and perfect shadows this would not be needed for specular. You can run into problem that the specular for some materials is so bright that AO is not enough Tri Arc proposed using a special specualr occlusion that has a much higher occlusion factor.

References Sébastien Lagarde ”Adopting a physically based shading model “,”Feeding a physically based shading model” Roy Driscol “Energy Conversation In Games” Fabien Giesen “Blinn-Phong normalization factor” Nathaniel Hoff­mann, “Crafting Physically Moti­vated Shading Models for Game Devel­op­ment”, SIGGRAPH 2010 Dim­i­tar Lazarov, “Physically-​based lighting in Call of Duty: Black Ops” SIGGRAPH 2011 Chris­t­ian Schüler, “An Efficient and Physically Plausible Real-​Time Shad­ing Model.” ,” The Blinn-​Phong Normalization Zoo” Akienne-​Möller, Haines and Hoff­mann, “Real-​Time Rendering” book Yoshiharu Gotanda, “Practical Implementation of Physically-​Based Shading Models at tri-​Ace”, SIGGRAPH 2010 Yoshiharu Gotanda, “Practical Physically ​Based Rendering in Real-time”, SIGGRAPH 2012

Testing on real projects Better Geometry term
Going forward Testing on real projects Better Geometry term Using the proper mipmapping for cubemaps Some kind of Diffuse- specular energy conversation We are in the lucky situation that we can run tests of this on a lot of projects carefully tuning and twaking and see what makes the artists life easier and what does not. Having a better geometry term is a no brainer for us. Even though it will mean having to handle normal map mipmapping differently with variance values as in lazarovs note on black ops lightning Nothing in these demos are done with the correct cube blurring because in the beginning the used cubemap gen and later they wrote their own blurring code. We have tested proper blurring and it does give a bit of an extra visual edge. Again tests has been made that are promising. This way artist can use albedo photo references more easily since we overide part of the issues in it by using proper valeus in the substance map) However we haven’t finished a project with this setting yet.

Physically based for non realistic rendering
Can we do it ? Yes just a material interface Rules can be bent and broken. Specular not that important for cartoonish look Diffuse light ,albedo,normal,AO more important Non realistic rendering Can we do it ? Yes, just a material interface Most of cartoony look comes from albedo,normal and AO. And diffuse of course

RTS game Here is an example of a semi realistic style running in PBL.
The reason this is important is of course so that we can keep giving the artists one interface that they are as familiar with as possible.

More non photo And here is another different view.
So the material interfaces is still flexible enough for any view while still keeping the interfaces low in fact the realism factor comes more from diffuse/normal and the design of the cube maps etcs.

Similar presentations