Presentation on theme: "Spectacular Specular -LEAN and CLEAN specular highlights"— Presentation transcript:
1 Spectacular Specular -LEAN and CLEAN specular highlights Dan BakerFiraxis Games
2 Motivation Observations from Movie folks “The most important aspect of rendering for a movie is anti-aliasing”“Games today still don’t look as good as animated movies 12 years ago.”“Why in video games does everything look so shiny?”Things I have heard over the years
3 Reality Check Have a habit of comparing ourselves against other games Therefore miss the obvious: Some of our materials don’t look anything like they shouldAnd they aliasAnd they sparkle
4 Shiny things Water, Metal, commonly seen in games Used for water in Civilization V, but metals suffer from similar problemsWill tour two common lighting models, Phong and Blinn PhongBoth have huge problems
5 Phong Simple to implement Since L is constant for environments, can turn specular part into a preconvolved environment
6 Sometimes accurateGood for perfect reflectors,like still water. But, perfect reflectors = high powerReflective materials have powers > 10000, but that will cause all sorts of problems for a point light
7 Problems with Phong Aliases Can’t get elongated reflections Pretty inaccurate – very plastic look to itNo good way to add normals maps togetherAnisotropic (grooved) materials will lose there anisotropy at zoomCan’t use high powers or else!
8 Problems with PhongOnly see a reflection on the very flat part of the beach, even with perfect use of Phong, won’t work.Classic Scene, sunset. Can’t get this with Phong lighting
9 Blinn Phong Much more accurate if we have real lights in our scene Can get elongated shapesCheap to evaluateNote, this is the normalized version of Blinn-Phong
10 Blinn Phong problems Aliases Shading becomes very wrong with roughness Highlights change based on pixel coverage, and becomeAnisotropic (groved) materials will lose there anisotropy at zoomCan’t do environments easilyCan’t use high powers or else!
11 Spectacular Specular fail Down sampled of thefinal renderWhat Blinn-Phong gives usA very rough wave filters to a perfectly flat standing water,An exampleThe effects are profound. If we take the same image and downsaple it, we get a very different thing then if we filter the normal map. Infact, we are being severely over shiny about things.
12 Bump FilteringConfirm with imperical data.The left side would look like the right side by most techinques. As we zoom this image out we notice massive differences in shinyness, even though it’s the same materialExactly the same substance, but one side is wrinkled. This completelychanges the reflections. (Thanks to Chipotle for the foil)
13 Overviews problems“The shimmies”, “The speckles” Lots and lots of talks about this problemThe more substantial the normal map, the higher the power, the more noise we get. Lots of artist tweaking, limits to our dataReflections are just plan wrong at distant scale, makes objects way over-shinyCan’t add normal maps together easily to get detail maps
14 Why this happensThe integral of a function over a range of inputs isn’t the same as function with inputs integrated over a rangeWhere R is a region of a texture, F is our shader (in this case, a Phong or Blinn Phong shader) . The second version is a discrete version, where W is the sample weights from our hardware filtering.Left side is what we want. This is the evaluation at every thoeretical texel that contributes to our scene. But, what we end up doing is the right side. As seen, these two are very different.
15 How do Movies solve this? Typically use REYES, or more advanced techniquesRoughly equivalent of shading every relevant texel and averaging the resultsVery expensive, potentially thousands of shader evals per pixelMovies typically decouple shading frequency and object rasterization. This gets rid of a huge number of issues, though still not completely ‘correct’. Likely will never do this for games, since objects small on screen can be hugely expensive.
16 Dreaming the dream Ideal lighting model: Can use any power we want Will deal with zooming in and out correctlyWon’t aliasEasy to use: compatible with our current pipelineRelatively inexpensiveCan add normals togetherCan use all of our MIP hardware
17 Formal definitionWhat we really want is to build a replacement for Blinn Phong that has this property (where F is basically our shader):We want a BRDF that is the same when we average all the results at the texel level with the results if we average the inputs. We cannot do the first one feasibly, but we can do the second (and it is already done via MIP maps and hadware).
18 LEAN Mapping Linear Efficient Antialiased Normal Mapping Considered Fast Antialiased Reflectance Texture MappingFast and flexible solution for bump filteringShiny bumps won’t aliasDistant bumps will change surface shadingDirectional bumps will become anisotropic highlightAllows blending layers of bumpsWorks with existing Blinn-Phong pipeline
20 Prior Work Posed by Kajiya 1985 Monte-Carlo Cabral et al. 1987, Westin et al. 1992, Becker & Max 1993Multi-lobed distributionsFournier 1992, Han et al. 2007Single Gaussian/Beckmann distributionOlano & North 1997, Schilling 1997, Toksvig 2005Diffuse [Kilgard 2000]Some previous attempts, some more worth mentioning
21 Beckman Shading ModelThe math is simpler then it looks. We are raising the power based on the distance of the half angle from the normal. This second formulation rolls the power into a covariance matrix, thereby giving us anisotropic power (e.g. two powers, one for X and one for Y).
22 Probability Distributions in Shading Distribution of microfacet normalsPerfectly reflective facetsOnly facets oriented with reflect toLook up probability of in distributionBeckmann distributionGaussian of facet tangents = projection
23 Filtering Filter is linear combination over kernel Linear representation → any linear filterSummed Area, EWA, …MIP map, Hardware AnisotropicWe need a BRDF that is linear
24 Filtering: Gaussians Gaussian described by mean and variance Mean combines linearlyVariance does not, but second moment does
25 Blinn-Phong ↔ Beckmann Blinn-Phong approximates Gaussian [Lyon 1993]Better fit as increasesVariance , normalize withBechmann is very close a normalized Blinn-Phong. We were using normalization factor of s for Blinn-Phong.
26 Blinn-Phong ↔ Beckmann Blinn Phong and Bechmann are nearly equivalantBlinn-PhongBeckmann
28 Distributions & BumpsIf the normal is changing our surface orientation, is there any way to add them together?Does that evenmeaning?
29 LEAN MappingBeckman distribution can be broken into pieces that filter, but doesn’t deal with the normals.Key insight: We think of the normal instead as a shift of the distribution of microfacets
30 DistributionsBeckman distribution works on a 2d plane. The blue discs represent the distribution of normals. Rather then change the orientation of the surface, we simply shift the center location of the distribution of normals by the x,y component of the normals. Thus, we interpret the normal as a shift in distribution, rather then a change in surface orientationThe green arrow represents the distance used by the beckman distrubtion. It is a vector in 2D, and thus the beckman distribution can be anisotropic since the x and y directions can have what is in effect different components.
31 Filtering Bumps Rather than bump-local frame Use surface tangent frame Bump normal = mean of off-center distribution
32 Surface-frame Beckmann Bumps vs. Surface FrameBump-frame BeckmannSurface-frame Beckmann
33 LEAN Data Normal (for diffuse) Bump center in tangent frame Second moments
34 LEAN Use Pre-process Render-time Seed textures with , and Build MIP chainRender-timeLook up with HW filteringReconstruct 2D covarianceCompute diffuse & specular per light
35 Sub-facet Shading What about base specularity? Given base Blinn-Phong exponent,Base Beckmann distributionOne of these at each facet = convolutionGaussians convolve by adding ’sFold into , or add when reconstructing
36 LEAN Map features Seamless replacement for Blinn-Phong Specular bump antialiasingTurns directional bumps into anisotropic microfacets
37 Bump Layers Uses Bump motion (ocean waves) Detail texture Decals Our approachConceptually a linear combination of heightsEquivalent to linear combination ofEven from normal maps
38 Bump Layers: The Tricky Part What about ?Expands out to , , and termsterms are in , terms are interms are new:Total of four new cross terms
39 Layering Options Generate single combined LEAN map Mix actual heights, or use mixing equationsTime varying: need to generate per-frameDecal or detail: need high-res LEAN mapGenerate mixing textureOne per pair of layersDecal or detail: need high-res LEAN mixture mapsApproximate cross termsUse rather than a filtered mixing texture
41 1600 x 1200, single full screen object PerformanceSingle LayerTwo LayersBlinn-PhongLEANPer-frameMix textureApproxATI Radeon HD 58701570 FPS1540 FPS917 FPS1450 FPS1458 FPSD3D Instructions30 ALU 1 TEX42 ALU 2 TEX50 ALU 3 TEX54 ALU5 TEX4 TEX1600 x 1200, single full screen object
42 Converting Blinn-Phong Data So fast could be done at load timefloat3 tn = tex2D(normalMap, coord);float3 N = float3(2*tn.xy-1, tn.z);float2 B = N.xy/(ScaleFactor&N.z);float3 M = float3(B.x*B.x + 1/s, B.y*B.y + 1/s, B.x*B.y)Output.lean1 = float4(tn, .5*M.z + .5)Output.lean2 = float4(.5*B + .5, M.xy)S is the power, ScaleFactor is a reranging factor
43 Texture Compression and Precision Normal maps get big, painful to compressLean MAPs require 5 fieldsx,y, x^2, xy, y^2Caveat: The precision matters. Unlike other techniques, we are using the normal filtering hardware
45 Typical strategy Remember that our is stored power is 1/s Simple normalized texture, pow 32 = 4 bits precision, pow 128 = 2 bits!Can renormalize range, to capture some bitsIf we want to use very high powers, e.g. 10,000+, really need 16 bits precision
46 Water For Civilization V Lots of background, but why did we do this?Needed to make water that worked at a distance, not a smooth reflectionAnd, wanted a realistic wave combing effectDoes not use a reflection map, high powers let us use an analytic model istead
47 Civ 5’s water Linear combination of 4 moving bump maps Allows us to accurate wave directions
48 Can we make a cheaper version? CLEAN MappingAn extension to LEAN mapping developed after paper publishedCommon art problem: Went to 5 values, hard to drop into most pipelines, and need more precisionCan we make it use less valuesCLEAN mapping Cheap Linear Efficient Antialiased Normal Mapping.
49 Dropping AnisotropyCool feature of LEAN maps, but efficiency might be more importantLet’s examine the Beckmann distribution againBe really nice if we could make only 1 value instead of 3
50 Dropping TermsCan just approximate the covariance matrix with a diagonal matrixThen store just X^2 + Y^2 in addition to X,YRemember that we aren’t after X^2 – Y^2, but rather the variance of it
51 CLEAN MappingNow we have only 3 terms to store. X, Y, X^2 + Y^2, can store in 3 valuesThen, calculating the variance:
52 Combining CLEAN MapsCombing two maps M1 and M2 with lerp factor t
53 Coming CLEAN Most of the high level benefits of LEAN mapping About half the data costsDoes not support anisotropy
54 Conclusions Normal map filtering = solved problem Cheap, easy to make art forHuge Visual ImpactNO EXCUSE to have messy specular!
55 ThanksMarc Olano – can find I3D paper on his websiteFiraxis Games