1. The “Dark” Side of Game Development
Shadowing in XNA

2. Let’s see some ID… Julian Spillane
CEO / Technical Director, Frozen North Productions, Inc.
We make video games.
Currently working on a next-gen XBLA project
January 2008 release
PC release to follow soon after…

3. Break it down… XNA? Shadowing in Games Shadowing Algorithms
How does it stack up
Reception by the pros
Shadowing in Games
Shadowing Algorithms
Maps, volumes, geometry projection…
Why I prefer maps
Basic Shadow Map Theory & Implementation
Variants
DPSM, PSM, TSM, LiSPSM, PSSM, VSM…
Tips for Optimization, Soft Shadows
Further Reading

4. XNA’s Not Acronymed What IS XNA? Who is using XNA?
A managed framework for developing games on both PC and Xbox 360
Graphics device, input, and audio management along with math helpers
Who is using XNA?
Hobbyists
Independent developers
… and yes, professionals
Torpex Games, Hidden Path Entertainment, GarageGames Inc.

5. XNA’s Not Acronymed (cont’d)
Why XNA?
Unmanaged background
C, C++, assembly
XNA exposed me to a new language
XNA allows for RAD
Full games can be put together in obscenely short periods of time
Allows developers to focus on content, gameplay, balancing over technical issues
XNA does things more inuitively than DirectX
Render Targets
Device Management
Preaching to the choir? ;)

6. XNA’s Not Acronymed (cont’d)
Full game built in 2 weeks using XNA with no prior knowledge of the API…

7. Shadowing in Games Why are shadows important?
Add a sense of presence and volume to a scene
Important visual cues
Make for interesting gameplay mechanics
Adds to realism (caveats?)
Players expect dynamic lighting
Shadows play a major role
How many different ways are there to shadow?
In a word: lots

8. Shadowing Algorithms Geometry Based Image Based Projected Shadows
Geometry collapse using “Shadow Matrix”
Shadow Volumes
Regular volumes
Z-fail / “Carmack’s Reverse”
GPU-generated
Both cases ~O(n) in terms of scene complexity
Image Based
Shadow Map
Depth-based shadowing system
Many variants: PSM, TSM, LiSPSM, PSSM, CSM, VSM, DPSM, etc.
Relatively O(1) in terms of geometric scene complexity
Forward Shadow Mapping
Proposed technique

9. Shadowing Algorithms (cont’d)
Geometry Based
Pros
Sharp, crisp shadows
Easy to compute
If it’s visible on-screen, you already have the data to shadow it
Easily extendable to the vertex shader
Speed benefits through pipeline-integration
Cons
Shadow complexity is O(n), n = # edges
Means speed α complexity-1
Fine balance between object count and edge resolution
Requires image-space modification or ugly jitter for soft-shadows
No support for alpha-blended textures without extreme trickery
We’re talking leprechaun-level, here…

10. Shadowing Algorithms (cont’d)
Shadow Volumes – Algorithm, Demonstration
Images courtesy of Østfold University College, Norway and NVIDIA Corporation

11. Shadowing Algorithms (cont’d)
Image Based
Pros
Shadow complexity α scene complexity!!!
Means cube equally expensive as Marcus Fenix ™ (sorta…)
Gives shadowed alpha for free
Soft shadows feasible in real-time
Self-shadowing is free
Cons
Inherently aliased / artifact’d
Aliasing from texture resolution and light projection settings
Artifacts from limitations of texture projection
Both are mitigated through technique variations
Memory intensive
Memory consumption α shadow quality
Dependent on texture size
No unified approach
Different techniques for different situations

12. Shadowing Algorithms (cont’d)
Images courtesy of me. Demonstration of vanilla shadowing using PCF and screen-space blur.

13. Shadowing Algorithms (cont’d)
This presentation will focus on shadow mapping, its theory, implementation and variants
On modern hardware, pros far outweigh cons
Self-shadowing and alpha blending!
Scenes in games are growing in complexity
Vertex-bound techniques will cause even more problems as time progresses
Independent of model / vertex data architecture
Personal preference ;)

14. What Is Shadow Mapping? Rendering from the light’s POV
Using results to generate projective texture
Generally filled with single-color depth information
Two-Pass Process
Generating the shadow map
Shadowing the scene
Shader Approach
Can be done using FF, but deprecated

15. Creating the Shadow Map
Theory
Render from light’s point of view
Any object that can be seen must be lit
Store the depth value of every element
Render the scene as normal
Take the position of every vertex
Transform into light-space coordinates
Retrieve the point as it is seen from the point of view of the light (depth map)
Depth-compare with light-view map
If depth value of surface is further away, it must be behind an occluder and shadowed (z-fail test)

16. Creating the Shadow Map
Can be represented in many formats
Best Choice: R32F (single channel float)
Only if hardware supports…
Can use ARGB, just less precision
Unless interesting depth hashing with the channels…
Use XNA RenderTarget2D
Writeable surface, can convert to texture
Easy to use

17. Creating the Shadow Map
Matrices
Need light’s view, projection matrices
Choice in projection
Ortho best for large, directional lights (i.e. sun)
LightProj = Matrix.CreateOrthographic(w, h, n, f)
w, h are width and height of view volume
n, f are near and far planes of the view volume
Recommend calculating these instead of arbitrary
Perspective for local, spot lights
LightProj = Matrix.CreatePerspectiveFOV(f, a, n, f)
f: perspective FOV, a: perspective aspect ratio
Implies we need light information
Position, target, up vector
LightView = Matrix.CreateLookAt(position, target, up)

18. Creating the Shadow Map

19. Creating the Shadow Map
Effect
Let’s now consider the effect
Needs to push unlit vertices
Return depth information
Store depth independently
POSITION cannot be read directly
Very simple shader
Don’t make it any more complex than it needs to be

20. Creating the Shadow Map

21. Creating the Shadow Map

22. Creating the Shadow Map
Statue seen from the light’s point of view. Blue because of R32F texture. Notice gradation: depth values

23. Shadowing the Scene Projection Now we have depth values stored
Need to render a second pass from camera
Project the shadow texture into the scene and depth compare
Yet another matrix
Texture bias and projection matrix
Used to map position from projection space into texture space

24. Shadowing the Scene Depth Comparison
Transform position using previous matrix
Compare z-value to value stored in shadow sampler
if stored depth < visible depth at that point
Must be occluded and therefore shadowed
Easy pixel shader comparison
Single texture lookup with 1 inline conditional branch (ps_2_0 only)

25. Shadowing the Scene

26. Shadowing the Scene

27. Shadowing the Scene
Images courtesy of Frozen North Productions, Inc.

28. Shadowing the Scene Final Steps Merging with the scene
Simply multiply shadow term by diffuse colour / lighting / texture
So…improvements?
Aliasing artifacts, projection artifacts, hard edges, the list goes on
How do we mitigate the problems?
Variations on the technique
I will cover overviews with links to papers and demos

29. Shadowing the Scene
Images courtesy of Frozen North Productions, Inc.

30. Shadow Map Variants Dual-Paraboloid Shadow Maps
Basic shadow mapping is inherently spot / directional
Need creative solution to handle omnidirecitonal (point) lights
Paraboloid Mapping
“[I]mage obtained by an orthographic camera view a perfectly reflecting paraboloid.”
Single map can cover a full hemisphere
Implies only 2 rendering passes for omni light vs 6 passes for cube lookup
Find a point P in R3 on the paraboloid that reflects a given direction towards +/- Z
Depth value calculated by using the distance from surface point to center of paraboloid
Extends from a 2D-3D mapping to a 3D-3D mapping
“Shadow Mapping for Hemispherical and Omnidirectional Light Sources” Brabec, et al., Computer Graphics Group, Max-Planck Institute, Saarbrucken Germany

31. Shadow Map Variants Perspective Shadow Maps
Aliasing due to depth resolution issues
PSMs
Generated post-perspective transformation
Light is transformed projectively to the unit cube
Reduces perspective aliasing since the depth values are considered after perspective proejction
Caveats
Cases to handle for objects behind the viewer but cast shadows (lit from behind the viewer)
Comparatively convoluted implementation
“Perspective Shadow Maps”, Stamminger, Marc and Drettakis, George, REVES – INRIA Sophia Antipolis, France

32. Shadow Map Variants Trapezoidal Shadow Maps Similar to PSMs
Approximates eye frustum as seen from the light using a trapezoid to warp it onto a shadow map
Designed to address perspective-induced aliasing and quality discontinuity (flickers)
Biases shadow map so quality increases closer to the camera
“Anti-aliasing and Continuity with Trapezoidal Shadow Maps”, Martin, Tobias and Tan, Tiow-Seng, School of Computing, National University of Singapore

33. Shadow Map Variants Light-Space Perspective Shadow Mapping LiSPSM
Modification to the PSM algorithm
A bit of a mouthful
Designed to address artifacts inherent in the PSM algorithm
Perspective distortion, missed shadow casters, singularities in post-projective space
Perspective transform specified in light-space
Allow for treating all lights as directional while still maintaining perspective benefits
Far more complicated than regular shadow maps
Benefits outweigh time cost of implementation
General opinion mixed
“Light Space Perspective Shadow Maps”, Wimmer et. al, Vienna University of Technology, Austria, 2005

34. Shadow Map Variants Parallel-Split Shadow Maps PSSM
Splits the view frustum into parts using plane parallel to the viewing plane
Generates smaller shadow maps for split parts
Tighter bounds on each map takes better advantage of texture resolution
Results in crisper shadows at lower cost
Widely implemented
Lots of available example code
Current shadow map fad
Very useful for large, outdoor environments
Makes good use of orthographic cameras
“Parallel-Split Shadow Maps for Large-Scale Virtual Environments”, Zhang et. al., Chinese University of Hong Kong, 2006

35. Shadow Map Variants

36. Shadow Map Variants Variance Shadow Maps VSM
Developed by a fellow UW alum and IGDA Toronto member
Andrew Lauritzen, University of Waterloo student
Presented at i3D 2006 then GDC 2007
Taking the industry by storm
Designed to solve aliasing issues by using calculated variance to determine an upper-bound on the fraction of an occluded, shaded fragment
Store mean and mean2 of a range of depths
Allows computation of variance
Relatively simple to compute on modern GPUs
A bit more overhead, but well worth the cost
Allows for mipmapping and texture filtering
Generally uncomputable for SSM because of single depth value
Caveats
Light bleeding on filter areas with high variance
“Variance Shadow Maps”, Donnelly, William and Lauritzen, Andrew, CGL, University of Waterloo

37. Shadow Map Variants

38. Optimizations Things to consider Texture size Conditional statements
Depth comparison in shader
Texture filtering
Combining of algorithms
PSSM + VSM particularly interesting for large outdoor scenes
Selecting the right algorithm for the scene
Unified solution does not exist

39. Soft Shadows Large Topic Various Techniques
Could spend another two hours on various techniques
Various Techniques
Percentage Closer Filtering
Penumbra Calculations
Screen-Space Blur
Combinations…
Will demo PCF with Screen-Space Blur

40. Further Reading
“Casting Curved Shadows on Curved Surface”, Williams, Lance, CGL, New York Institute of Technology, New York, 1978
The original introduction of shadow mapping (yes it’s that old)
"Shadow mapping." Wikipedia, The Free Encyclopedia. 1 Sep 2007, 09:57 UTC. Wikimedia Foundation, Inc. 29 Sep 2007 <
“Shadow Mapping Tutorial”, Paul’s Projects,
“Hardware Shadow Mapping”, NVIDIA Corporation

41. Questions?

42. Thank You
Contact Information
Office: (519)
Website: