Presentation on theme: "Physically Based Shading"— Presentation transcript:
1 Physically Based Shading A simplified artist interfaceNiklas HanssonHead Teacher Game programming The Game Assembly.
2 Physically Based Shading What is it ?Why should I use it ?How do I integrate it into a project ?Lightning base on the physical properties of light.Replacement of Ad Hoc solutionsInstead of havingSpec Intensity,Spec Gloss,Reflectivity,Fresnel strength map and even a separate reflection gloss factor.With a physically based solution we will be able to collapse this down to two simple maps.Instead of Ad hoc solutions competing we have one stable system.
3 Boundary : When light goes from one material to another Basic BRDF TheoryBoundary : When light goes from one material to anotherLight splits up into two directions : reflection andrefraction.The amount of lightreflected depends on thematerials reflective index.Been asked to keep this not to technicalBasics are important.BoundraryL is split into ri and tSplitRefractive indexBRDFWorks no matter the materialsWe will only look at air to other materialImage from “Real-Time Rendering 3rd Edition”
4 Basic BRDF Theory Basic Material - Part is Reflected - Part enters the materialMetalPart is ReflectedThe part entering theMaterial is absorbedImage from “Real-Time Rendering 3rd Edition”
5 Basic BRDF Theory Non metals - Light that enters the material scatters internally- Is absorbed- and often exits the surface again.Image from “Real-Time Rendering 3rd Edition”
6 Sub surface scattering Basic BRDF TheorySub surface scatteringDistance between enter and exit positions are determined by the material.If red pixel and green exit posneeds SSSBRDF cant cope.If other wayWorksImage from “Real-Time Rendering 3rd Edition”
7 Basic BRDF Theory Point model By ignoring distance between exit and enter points we can model lighting with an BRDF.Holds holds true no matter the source of the light.Works for perfectly flat boundariesMost real world materials (except mirrors) doesn’t share this property.Image from “Real-Time Rendering 3rd Edition”
8 Basic Microfacet BRDF Theory MicrofacetsA surface consists of many small perfectly flat surfaces.How much they differ in normal determines how light interactMicrofactesNormal differences encoded as a normal distributionThe difference from the Phong model to the Blinn Phong modelBlinn Phong instead used the microfacetIf parallel light hits small solid angle with all light, the rougher the larger solid angle the light will hit.We use a normal distribution function to model thisThe data it uses is the roughness of the surface (or flattnes would be a better name)Image from “Real-Time Rendering 3rd Edition”
9 Basic Microfacet BRDF Theory As you can see the rougher the surface the blurrier reflection and larger specular highlights.Microfacets model sub normal map resolution shapesImages from “Real-Time Rendering 3rd Edition”
10 Shadowing & MaskingNot all micro facets facing half vectors direction contribute to lightningShadowingMaskingStrength of being physically basedValues have real valueCan update formulas without changing the valueCan be used to uppdate difuse to for exampel to Oren-NayarRough material not gonna be as reflective due to ridges blocking lightMicrofacet BRDF cant handle bounces on ridgesImages from “Real-Time Rendering 3rd Edition”
11 So why is specular so important ? Images from “Everything is Shiny” by John Hable
12 So why is specular so important ? Images from “Everything is Shiny” by John Hable
13 The Microfacet BRDF Microfacet BRDF LEGO parts Most different BRDF models only contains changes to the D and G term.Except denominator//Is a correction factor that accounts for quantities being transformed between the local space of the micro facets and the overall macro surfaceThe ‘4’ is related to transforming from the space of half-angle directions to other spaces,and the two clamped dot products are foreshortening factors that relate to the way that the visible area of microfacets change in relation to their orientation.
14 Returns a Value from 0-1 in RGB Fresnel ReflectanceReturns a Value from 0-1 in RGBFraction of light reflected from optically flat surface given Light directon l and Half vector hFrist part Fresnel reflectivityGives the reflctivity of the surface vs the viweing and light angle.Returns spectral valuesReflected color doesn’t absorb lightHowever they reflect different amounts of lights at different wavelengthsWe simplify this using RGB just as in the rest of CG
15 Fresnel ReflectanceDepends on refraction index (in our terms the substance of the object) and the light angle.Fresnel curveLight bounce amount varies with angle.Try putting your head next to a door and se a clear reflection appear.Starts on index of refraction which we will call substanceObjects with RGB spec has 3 index of refractionBarely changes in beginning.Turns quickly upwards at the endSome materials dipWe wont model thisImage from “Real-Time Rendering 3rd Edition”
16 Fresnel Images from “Everything has Fresnel” by John Hable Some real world examplesImages from “Everything has Fresnel” by John Hable
17 Schlick’s Approximation Decently Accurate, uses F(0) the refractive index as input.Which is our substance term.
18 Gives a scalar value between 0 and 1 Geometric termGives a scalar value between 0 and 1Contains the fraction of non obscured microfacetsVarious functions exists most using roughness as inputUsing the geometric term to cancel out the denominator is extremely cheap as you can just remove booth from the calculation but will result in a to dark specular.Look at lazarovs keynote for good solutions to this.
19 Distribution function Calculates factor of microfacets reflecting light towards the eyeThe function determines size and shape of the specular highlightDistribution function calculates how many microfacets reflects light based on the roughneesDecides specular size and strengthThe blinn-phong distribution function is widely used in the games industry
20 Blinn Phong Distribution Functions Modified Blinn PhongBasic Blinn PhongDoesn’t account for amount of light hitting the surfaceUsing a max cap is not enoughCalcualte amount of light hittign surfaceAnd multiply with thatCant reflect more light than hits it.
21 Beckman Distribution function Conversion :Cook torrence used beckamanMore accurateMore expensive.Can convert between
22 Blinn Phong Energy Conversation If the specular highlight is larger the specular intensity must be lower.N = roughnessA surface cant emit more light than hits itTherefore of light is reflected over a larger solid angle.It needs to be weakerWe use a normalization factor to adapt light strength to scale.This makes the equation energy conservingThe normalization factors differs depending on the distributionN= roughnessphong has a very different normalization factor than blinn Phong.This normalization was derived by Fabian Giesen
23 Blinn Phong Energy Conversation Formula from physically based shading 2010Seems vey off ?Pi is lackingComes from hemisphere integrationApplies constantly over all light so can be removed in our case.Just change light scale with pi
24 Blinn Phong Energy Conversation Adding in the pi seems to give a correct curve so lets zoom in to make certain
25 Blinn Phong Energy Conversation Zooming in it seems that this curve is off with quite a bit in the lower 100 range at least.
26 Blinn Phong Energy Conversation Another common approximation as an 8 instead of a 2 at the top. This one fits much more well.An important part is that it fits better at lower values which contains most of the real world materials. So we get some extra precision where it counts.The factor of 8 is also used in the real time rendering book.
27 Blinn Phong Energy Conversation This is another common approximation taken from tri Arcs papers but as you can see it plots vs the n+2/8 curve and not the real one
28 Blinn Phong Energy Conversation This is another common approximation taken from tri Arcs papers but as you can see it plots vs the n+2/8 curve and not the real one
29 Blinn Phong Energy Conversation Mathematically actually this formula is a better match but as you can see we have a higher margin of error on the lower values where most natural objects resides which is why the formula with an 8 is more often used.
30 Our Normalized equation for specular light Energy ConversationOur Normalized equation for specular lightLambertCdiff*PI <=1Full BRDFCdiff+Cspec <=1Our equationLambert equation for diffuseUsing Cdiff for albedoTo be energy conserving Cdiff*pi<1Putting Cdiff and Cspec together the rule is that no material can reflect more light than hits it,This means albedo plus frefractive index need to stay below 1
31 Ambient Light is just Light Has a specular component tooreflectionsroughness controls reflections blurriness substance controls reflection strength.Pre blurred cubemapsAmbient is lightHas specularIs handled as reflectionNo need for extra data as we can already caluclate reflected lightIf not using ansitropic look up you can preblur mip map levels and use roughness as a curve.For spec we use a logaritmic curve to use as much a sthe color space as possible in the range the artist worksFor Mip map select we use the default roughnees to keep more mip maps in the just slightely blured part.
32 ATI cubemap gen has nice functions for preblurring mipmaps. Ambient LightATI cubemap gen has nice functions for preblurring mipmaps.For correctnes the blurring needs to match your specular lightPrefiltered cubemap will start start looking as diffuse lightCube map gen can prefilter mip maps.Doesn’t do it correctly for thisCan roll your own or just download and modify
33 Blurred cubemap Use the angular extent and always sample from top mip. Use the glossiness for that mipmap level and evaluate max size of highlight for size of angular extent.Use the phong function to evaluate a contribution for all the pixels inside the extent.Cubemap gen uses angular extents with different functions.We want is filtered in the same way as phong (cant use blinn phong as it’s anisotropic) but phong is perfectly roundCalculate angular extent per mip using roughnessUse phong to evaluate contribution for sample.NormalizePhong simplifies to cosine filtering . Already in Cubemap gen raised to the roughnessPhing normalisation can be ignore as its the same for all samples with the same roughneesDoing this makes specular highlight size and reflection bluriness match.
34 Why physically based ?It’s more realistic thanks to being physically based?Materials does not require as much retweaking under changing lighting conditions.You can create more easily varied materials with lesser number of valuesIt saves artist time and simplifies workflowThe top one here isn’t one that I worry about. It might be to some people but I am a pragmatic.Materials that doesn’t need retweaking as much when lighting changes is a good thing and saves time but by itself it is not a seller.The more varied material interface and the easy to create different materials with a low number of values is a big seller especially combined with deferred renderingThe last one here however is the seller for me. Saving artists time is just pure financial sense and the reason we call this a simplified artist interface and that’s the issue I with start with.
35 Simplified Artist interface Reduces number of maps from 4-5 to 2.Simplifies Junior artist trainingNo longer take albedo and desaturate for intensityAnd then lower color for glossWe have all seen itReduces maps vs ad hoccReduces work amountSimplifies artist mental modelA lot of erroneus tutorials around just using filters.Some tutorials talk about taking the inverse of albedo instead.Makes spec depend on albedo brightness not materialBrigher gets brighter or darker gets darker,Gloss not really understood just lower colorReflection just for special objectsFresnel no clueWill look like above.
36 Simplified Artist interface PBL simplifies this by creating a material interface that makes sense.SubstanceIs the index of refraction for the materialNo need to hand paint.Mathematically matches old intensityA interface you can understand.Using existing physical valuesUsing a look up table and masks to select areas and add materialMakes substance quick to createDoesn’t need hand paintingMatches old intensity
37 Color picker Easy visual look up Handles specular color properly Linear and gamma depending on what the project usesNon metals taken from physics books.Metals index for different wavelengths taken from scientific dataFeed into a program by sebastian lagarde that integrates them for RGB color space.Here is the swatch we use at The game assembly. We have booth linear and gammFor average materials default values at the bottomLast values to be tweaked.Else you might tweak others values for an incorrect here and never get it right.
38 Simplified Artist interface RoughnessThe old glossBut thanks to Normalization a lot more usefulWhere hand painting happensControls reflection blurrinees and specSurface Roughness that normal maps can’t showTouchObserveJust old gloss with normalizationCan make brighter specular thanks to normalisationMakes ideal for hand paintingControls reflection blurrines as wellArtists doesn’t need to worryHe only needs to think about the roughness of the surface on the level that won’t shop up in the normal mapsUsing touch to get a feelLooking for fresnel effects at glancing angles.
39 Example Material rusty Paint Substance just a mask for materialRoughness contain details
40 Flexibillity Don’t forget albedo and normals importance We made a test chart with some different materials to test it out and se how different looks we can achieve.
41 Sample Oskar stålbergThis is all rendered in our maya plugin for PBL and is a good example of how flexible material looks you can get with this system.Look especially at the mush rooms and metal vs the main body to the leftAnd the fuse and plastic cables to the right
42 Substance almost no detail except for big material change Roughness contains the workTried on junior and more senior artistsSenior directly grasped and improved speed.Juniors took a while to make the shift then increased speed
43 Lower numbers of values to tweak Smaller tighter gbuffer Why physically based ?Lower numbers of values to tweakSmaller tighter gbufferLow bandwitdhSmall ALU increaseIdeal for deferred renderingWhile still allowing veryvaried materialsAnd now the technical reasonsHaving fewer variables reduces bandwidthEspecially when using deffered renderingPBL works for almost all materials right of the bat so you need very few special cass-We get a tighter gbuffer for just a small ALU increasel
44 How to use it What did we do ? Microfacet BRDF with Normalized Blinn PhongFresnel reflectanceVoid Geometry FunctionG(h) = 4(n dot l)(n dot v)Prefiltered glossy reflections from cubemap probesAmbient from ambient probes
45 What we did Cref is the value from the reflection probe Camb is the value from the ambient probeFschlick(v,h) is the fresnel term using the view direction instead of the light directionRemoved the pi division to just change the light intensity scaleCref is the reflected light in the cubemapCamb is the ambient light for the point.fresnel term for the reflection uses v and h instead of l and h.energy conversation between reflection and ambient.while not 100% correct it does a good job of mimicking real life behaviorOther are using a special case reflection fresnel to reduce edge shine.This worked well for us.
46 Ambient vs reflection Energy Conversation Energy conversation created better metalic surfacesWhile still allowing artists to use an albedo texture.Effect is subtleRight side is more metallicLeft side seems washed out.
47 Also solved issues where artist tried to tweak substance and roughness incorrectly To remove to strong fresnel effect.
48 Substance and Roughness vs Intensity and Gloss Lessons learnedSubstance and Roughness vs Intensity and GlossSpend time training and communicating to your artistsHelp them use the new interfaceTools is essentialBeware of bugsYou got to have artist buy in.Need a mind shift.New names helped.We created a maya plugin to allow the artists to preview models.And had some issues due to bugs in it
49 Maya plugin Tool was double edges sword. Artist used new interface Maya visuals differed from game visuals
50 Reflection and ambient is a big part of PBL In game they are dependant on light conditions so that their strength match light strength. Using in engine probes.In maya the artist could select maps and tweak light to values that could not exist together in game.Which meeant what they saw in game and engine differed.A solution would be to try and normalzie cubemap vs light strength but getting 100% is imposible.We are looking at using premade light rigs to solve the problemUsing an in engine viewer is even better but artists doesn’t like the extra step of leaving maya.
51 Flexible Kind of projects Space shooter Realtime Strategy What we didFlexibleKind of projectsSpace shooterRealtime StrategyFirst Person shooterUsed the techinique in 12 projectsOver 4 teamsAnd 3 genres
52 Space shooter Space shooter was perfect case Ambient and reflection from skybox.Only thing we would change is use ambient vs reflection energy conversion
53 RTS games showed up the pipeline problems with maya vs ingame And to strong fresnel due to lack of energy conversation caused a lot of issues.
54 Fps project Goals Techniques Resolve all issues found during the last projectGet an easy to light solution that took little rebuild time and would work with just a few spot lights.TechniquesAll enviroment and ambient probes generated by the ingame engine to achieve consistency.Box projected cubemaps was used for reflections and a variation was used for ambientSimple Multi bounce solution for lightning
55 Box projectedion made cubemap lookup ansitropic would require importance sampling and multipel filtering to look weel.Solution was to interpolate between box proejcted look up and ordinaryIn engine generated ambient and reflection resolved earlier issues
56 Specular and reflection doing more work OcclusionSpecular and reflection doing more workWill look weird in areas normally don’t reach by much lightUse Ambient occlusionMultiply to specMultiply to reflectionIf needed add special Specular occlusionSince booth specular and reflection is doing a lot more workWe get problemsDark crevices shines up due to reflection.Made worse by the fact that reflection is added to normal color makes reflection in dark creavices overly shiny.A solution is using ambeint occlusionHolds the amount of direction that light can hit the surface from. The more hid the darker.Will keep your crevice dark.With everything as geometry and perfect shadows this would not be needed for specular.You can run into problem that the specular for some materials is so bright that AO is not enoughTri Arc proposed using a special specualr occlusion that has a much higher occlusion factor.
57 ReferencesSébastien Lagarde ”Adopting a physically based shading model “,”Feeding a physically based shading model”Roy Driscol “Energy Conversation In Games”Fabien Giesen “Blinn-Phong normalization factor”Nathaniel Hoffmann, “Crafting Physically Motivated Shading Models for Game Development”, SIGGRAPH 2010Dimitar Lazarov, “Physically-based lighting in Call of Duty: Black Ops” SIGGRAPH 2011Christian Schüler, “An Efficient and Physically Plausible Real-Time Shading Model.” ,” The Blinn-Phong Normalization Zoo”Akienne-Möller, Haines and Hoffmann, “Real-Time Rendering” bookYoshiharu Gotanda, “Practical Implementation of Physically-Based Shading Models at tri-Ace”, SIGGRAPH 2010Yoshiharu Gotanda, “Practical Physically Based Rendering in Real-time”, SIGGRAPH 2012
58 Testing on real projects Better Geometry term Going forwardTesting on real projectsBetter Geometry termUsing the proper mipmapping for cubemapsSome kind of Diffuse- specular energy conversationWe are in the lucky situation that we can run tests of this on a lot of projects carefully tuning and twaking and see what makes the artists life easier and what does not.Having a better geometry term is a no brainer for us. Even though it will mean having to handle normal map mipmapping differently with variance values as in lazarovs note on black ops lightningNothing in these demos are done with the correct cube blurring because in the beginning the used cubemap gen and later they wrote their own blurring code. We have tested proper blurring and it does give a bit of an extra visual edge.Again tests has been made that are promising. This way artist can use albedo photo references more easily since we overide part of the issues in it by using proper valeus in the substance map) However we haven’t finished a project with this setting yet.
59 Physically based for non realistic rendering Can we do it ?Yes just a material interfaceRules can be bent and broken.Specular not that important for cartoonish lookDiffuse light ,albedo,normal,AO more importantNon realistic renderingCan we do it ?Yes, just a material interfaceMost of cartoony look comes from albedo,normal and AO.And diffuse of course
60 RTS game Here is an example of a semi realistic style running in PBL. The reason this is important is of course so that we can keep giving the artists one interface that they are as familiar with as possible.
61 More non photo And here is another different view. So the material interfaces is still flexible enough for any view while still keeping the interfaces low in fact the realism factor comes more from diffuse/normal and the design of the cube maps etcs.