1. Global Illumination using Photon Ray Splatting
EG 2007, Prague
Global Illumination using Photon Ray Splatting
Robert Herzog* Vlastimil Havran‡ S. Kinuwaki*
K. Myszkowski* H.-P. Seidel*
*MPI Informatik ‡Czech Technical University Prague
Germany Czech Republic

2. Motivation:
Filling the gap between photon density estimation and final gathering
Photon Density Estimation
Photon Ray Splatting
Low-frequency noise
Overestimation at corners
Darkening near boundaries
Direction undersampling for glossy BRDFs
Mention gap between final gathering and photon density estimation
Washed out illumination for high-frequency geometry

3. Related Work
Ray Maps for Global Illumination, [Havran et al., EGSR 2005]
Gathering of photon rays for a disc in the tangent plane
Eliminates boundary bias
Complex and memory demanding search data structures
Scalable Photon Splatting for Global Illumination, [Lavignotte el al., Graphite 2003]
Efficient image based photon splatting approach with adaptive bandwidth selection
Restricted to triangle meshes and only diffuse light transport towards camera
Path Differentials and Applications, [Suykens et al., EGSR 2001]
Designed for antialiasing and adaptive radiosity tessellation
Radiance Caching for Efficient Global Illumination Computation, [Křivánek et al., IEEE TVCG 2005]
State-of-the-art for (ir)radiance caching on diffuse and glossy surface
Search k-nearest rays in the neighborhood  uses lazily constructed tree over spatially sorted rays (requires coherent search queries)
Mention paper EG2005 by pascal Gautron  radiance cache splatting…

4. Density Estimation Metrics
Standard photon density estimation on surfaces [Photon-maps, H.W. Jensen ‘01]
Photon density estimation for a disc in the tangent plane [Ray-maps, Havran et al. EGSR ’05]
Cosine-weighted photon density estimation in ray-space [Eurographics ’07]

5. Basic Derivation Irradiance from incident photon hit points:
Irradiance from incident photon rays:
( i.e. use directional information of incident flux )
E(x) = L(x,ω) cosθx dσ(ω)
The theoretical part
E(x) = L(x,ω) cosθx dσ(ω)
dΦ(x,ω)
dA(x) cosθx dσ(ω)
d2Φ(x,ω)
cosθx dσ(ω)
dA(x) =
dA┴(x,ω)
d2Φ(x,ω)
cosθx =
ΔΦi (xi ,ωi)
ΔA(x)
Kh(ri)
ΔΦi (xi ,ωi)
ΔA┴(x,ωi)
Kh(ri)
cosθi

6. Ray Splatting instead of Gathering
Efficient gathering of photon rays intersecting a sphere needs complex search data structures (e.g. ray maps [Havran et al. 05])
Our solution: photon ray splatting instead of gathering
Boils down to search over point data
Kh(r)
Shaded area represents splatting footprint (shown in 2D)

7. Choosing the Bandwidth (Splat-radius)
Photon density estimation needs adaptive bandwidth selection
Various statistical methods for bandwidth selection exist – though often too complex to be used for efficient photon density estimation
Common k-nearest-neighbor (k-nn) method is computational intensive
Either costly search for each density estimation query (gathering methods)
Or costly precomputation of the k-nn radius per photon (splatting methods)
Remedy: exploit knowledge from photon sampling, i.e. choose a photon’s splatting radius according to photon path probability density
We get this almost for free !!!
Bandwidth selection:
- Quite a difficult and costly problem
K-nn DE requires a priori knowledge about local photon density
Besides, k-nn density estimation only equalizes the noise level

8. Photon Path Probability Density
· g(x0, x1)
· g(xi-1, xi)
p( X ) =
pe(x0, x1)
ps┴(xi-1 , xi , xi+1)
direct light (1 bounce)
indirect light (>1 bounces) x0
pe(x0, x1) ~ Le ·cos(θ0 →1)
g(xi, xj) ~ cos(θj←i)/||xi-xj||2
ps┴(xi, xj, xk) ~ fbrdf (xi, xj, xk)·cos(θj→k)
Probability density depends on light source sampling density p_e
BRDF sampling density p_s , which is roughly proportional to the BRDF function,
and the “geometric density” θ1 x1 θ2 x2

9. Adaptive Bandwidth Selection
Choose the photon bandwidth inversely proportional to its path probability density
Reliable estimate only for a small number of bounces (1 to 2)
Need to slightly modify formula to become robust
See paper for details!
Color-coded visualization of the bandwidth (shown as photon splats)
Choosing bandwidth proportional to the inverse photon path probability density yields often similar results as k-nn density estimation
However still problems with light leaks  bandwidth does not adapt to gradients or better second derivative of the irradiance
For robustness, we clamp the squared distance term (do not allow too small length) and
Also reduce the sensitivity to path probability density for indirect light bounces
Note the elongated drop-like splat footprint for rays with grazing angles (blue splats near ceiling)
2. bounce
4. bounce
1-6 bounces
Indirect Illumination

10. Efficient Nearest Neighbor Search
Nearest neighbor search for ray splatting is simpler than for ray gathering (e.g. ray maps [Havran et al. EGSR05])
Boils down to a search for point samples in a cylindrical bounding volume
We use a standard kd-tree constructed on top of the eye samples with axis-aligned spatial splitting planes
We also experimented with a uniform grid (does not scale well especially for larger bandwidths  many empty cells are touched)
The search in the kd-tree is too conservative  in particular for large photon bandwidths
Better solution -> [Herzog et al. Pacific Graphics 2007]

11. Search in the Spatial kD-Treetb R t+ t- ta
tb > t-
ta< t+
!tb > t-
ta< t+ t-
leaf R R tb ta
Eye sample array
Animation to show the traversal and “volumetric” search in the kd-tree
Search buffer
,(ta ,t+)
stack

12. Moderately Glossy Surfaces
With ray density estimation we gain more information about the direction of incident flux
since more photons contribute to a surface point
no matter what is the size and orientation of the surface
Means: less noise in the directional domain at grazing angles
However, sampling is view-independent!!!
View-dependent importance sampling of BRDF (future work)
Furthermore, variance in the estimate is independent of the incident ray direction
Ideal for glossy BRDF evaluation (often larger at grazing angles)
2. Statement should be proved or justified!!!
Because of view independence important directions might be undersampled
-> would need multiple importance sampling

13. Glossy Light Field Representation
We tested 2 different methods for capturing the incident radiance field
Accumulating flux in discrete directional strata (uniformly distributed)
Projecting the incident flux onto a spherical basis (spherical harmonics) (all BRDF data is precomputed in spherical harmonics basis)
ΔΦ(θ1,φ1)
ΔΦ(θ0,φ0)
dA┴(x,ω)
dΦ(x,ω)
cosθx Ylm(ω)
λlm(x) ≈
ΔΦ(θ1,φ1)
ΔΦ(θ0,φ0)
s(θ0,φ0) →S0(x) +=
s(θ1,φ1) →S3(x) +=
* Kh(r1) cos(θ1)
* Kh(r0) cos(θ0)
λlm(x) +=
Ylm
(θ1,φ1)
(θ0,φ0) *
* Kh(r1) cos(θ1)
* Kh(r0) cos(θ0)
ΔA┴
ΔA┴
ΔA┴
ΔΦ(θ1,φ1)
ΔΦ(θ1,φ1)
Global ray directions -> icosahedron subdivision  20 global radiance images (each image capturing flux density for a discrete global solid angle)
-> filter each of these radiance images using discontinuity preserving image space filtering
Local coordinate frame aligned SH basis functions (filters in directional domain)
we use 4 bands
ΔΦ(θ0,φ0)
ΔΦ(θ0,φ0) S3 r0 S0 r1

14. (Ir)radiance Caching
Computation of (ir)radiance for each pixel is still too slow
However, irradiance function is really smooth due to low- pass filtering via density estimation
 sparse sampling of the (ir)radiance is sufficient
We propose an algorithm similar to cache splatting [Gautron et al., EGSR ’05] in the spherical harmonics basis
Cache interpolation gives additional filtering of noisy photon splats  reduce overall splat radius  more efficient ray splatting

15. Radiance Cache Splatting
Noisy input from ray splatting
Adaptive radiance cache splatting
BRDF modulated and filtered result
Example of the cache splatting pipeline (runs on CPU)
Exaggerated example (not good to use irradiance caching for direct light)  only for demonstration purposes
Example shows pixel radiance (actually input is just irradiance or radiance field at the cache positions)

16. Changes to (Ir)radiance Caching
Due to view-independent ray splatting, we require a multi-pass cache splatting algorithm
For filtering purposes use smooth weighting function (e.g. bi-weight)
 More than one cache records must contribute to a pixel
Cache density and thus the filtering depends on the geometric gradient estimated using the split-sphere model [Ward ’88]
Ward ’88
Bi-weight
Cache splatting assumes coherence in the eye samples (not necessarily given for indirect eye samples  too dense sampling!!!)
Estimate harmonic mean distance from the reciprocal length of neighboring ray splats
Estimating harmonic mean distance Rc at caches r1 and r2 from projected ray distances (z1,2) to the ray origins (red circle).
-a·Rc
a·Rc
Cache weighting functions

17. Multi-Pass Radiance Cache Splatting
Cache locations (red) + cache weights (~grey shade)
Upper images: cache weights and position
Lower images: image after cache splatting (not showing the correct illumination  only for demonstration of the refinement)
Cache splatting result
Pass 1
Pass 2
Pass 3

18. Results: Diffuse Sibenik Scene
500,000 indirect diffuse photons
512 x 512 image resolution
Compared against reference (photon mapping + final gathering)
Photon density estimation (500 k-nn)
Photon ray splatting
Ray splatting with caching and filtering
Final gathering (1200 rays per pixel)
20 sec
35 sec
29 sec
~4000 sec
photons in total
Photon ray splatting tuned to have a similar bandwidth on average compared to 500 k-nn

19. Results K-nn Photon Density Estimation
10001000 pixel, photons
500500 pixel, photons
10001000 pixel, photons
Results
K-nn Photon Density Estimation
69 sec (400 k-nn)
66 sec (600 k-nn)
92 sec (400/ 250 k-nn)
Photon Ray Splatting (with radiance caching)
Comparing k-nn photon hit density estimation and ray splatting
Same scene setting similar bandwidth on average
Total times including ray tracing, photon shooting,…
Times in brackets are caching times
63 sec (16 sec)
64 sec (20 sec)
52 sec

20. Two-Pass Final Gathering
Path-traced (2000 rays)
Conference Scene 4700700 pixels, photons
Indirect illumination conference ( photons)
Without irradiance caching,  timings are about 2 orders of magnitude slower
Photon mapping (70 k-NN)
Path tracing (2000 paths per pixel)
Ray splatting (600 rays) ~ 64 min
Photon mapping (70 k-nn 600 rays) ~ 85 min

21. Conclusions
Ray density estimation as trade-off between photon density estimation and final gathering
No boundary bias, no topological bias
Works well with (ir)radiance caching
However, still problems with proximity bias (light leaking)
requires occlusion testing (future work)
(see [Herzog, Pacific Graphics ‘07])
only low-frequency (indirect) illumination can be rendered so far
More details provided in a technical report (coming soon)

22. Future Work Improving adaptive cache splat size computation
Make use of illumination gradients and density estimation variance (estimated from the number of photon splats per cache) to compute a better cache splat size
Temporal domain
Reuse caches for future frames and filter in temporal domain
Detect occlusions during photon ray traversal

23. Occlusion Testing
camera
camera
Current splatting neglects occlusion in the footprint
Correctly masking the density estimation kernel

24. Unpublished Extensions: What about Occlusion?
Ray Splatting
Visibility Ray Splatting
Lightcuts [Walter 05]
(reference)
Direct light (50,000 photons/ VPLs)
Conference Scene, (1000 x 1000 eye samples)
26 sec
30 sec
675 sec (~245 rays)
Global Illumination (250,000 photons/ VPLs)
Using gathered eye sample position to mask the kernel (see previous slide)
(produces wholes if eye sample density is too low)
Only works for direct occluders that are visible from the camera
85 sec
104 sec
1070 sec (~339 rays)

25. Thank you

26. Clamp squared distance term in path probability density
(Additional Material – Not presented at EG ’07) Bandwidth Selection: Practical Issues
Clamp squared distance term in path probability density
Two user parameters to control bandwidth:
Overall noise  bias level ( C )
Variance in bandwidth ( S ), i.e. sensitivity to path probability density (S=0 : constant bandwidth)
For convenience S should be individually chosen for direct and indirect light
Direct light ( S ~ [ ] )
Indirect light ( S ~ [ ] )
Bandwidth automatically adapts when number of photons changes
(proportional to 6th root of photon number)

27. (Additional Material – Not presented at EG ‘07) Cache Weighting Functions:= :=
Ward ‘88
Clampled Ward :=
Smoothed Ward
Biweight :=
Various weighting functions we have experimented with (used the last one in the end)
wc(x) := cache weighting function
Rc := harmonic mean distance
a := user defined cache-error
nc := cache surface normal
n := surface normal at position x
-a·Rc
a·Rc

28. (Additional Material – Not presented at EG ‘07) Irradiance Gradients for Ray Splatting
Angle θi with surface normal does not change in our metric (ray’s direction is constant)
(xi ,ωi)
ray.dir
r(x,xi,wi) Vx x
Very cool an interesting extension to the ray splatting (works well even for direct light shadows)
Of course small lighting features which are undersampled (e.g. conference scene -> chairs) cannot be detected by the gradients either! Ux
Kh is the Epanechnikov kernel function.

29. (Additional Material – Not presented at EG ‘07) Ray Cache Splatting with Irradiance Gradients (i.e. sample denser and thus filter less where the gradient is high)
Estimated irradiance gradient magnitude from ray splatting
Caveat: use gradient only for determining sampling density, not for cache interpolation!!! (gradients are too noisy)
 Sharpen shadow boundaries
Cache splat result without gradients
Cache splat result with gradients
Difference image (color coded)
(cache splatting for direct + Indirect Light)
Interpolated irradiance
( photons)
Results when applying gradients (simple test scene with area light in the neighbor room).
(Sphere is coarsely tesselated!! No rendering artifact!!!)
5.4 sec
5.7 sec

30. (Additional Material – Not presented at EG ‘07) Scalability of Photon Ray Splatting
Ray splatting only (directly to pixels) (RS direct)
k-NN Photon density estimation (PM k-NN)
Ray splatting to the directional histogram (RS histogram)
Ray splatting + SH radiance caching (RS+SH caching)
Ray splatting time to SH radiance caches (RS only)
photons
500 x 500 pixels
Some graphs which are also in the technical report! Is not up-to-date anymore (caching is more efficient now!)