1. Inverse Volume Rendering with Material Dictionaries
Ioannis Gkioulekas1
Shuang Zhao2
Kavita Bala2
Todd Zickler1
Anat Levin3
1Harvard
2Cornell
3Weizmann

2. Translucency is everywhere
food
skin
jewelry
architecture
We deal with translucency in many aspects of our everyday life. Our food and skin are translucent; and so are objects as diverse as jewelry and even buildings.

3. Subsurface scattering
outgoing direction
incident direction
isotropic
extinction coefficient σt(λ)
scattering coefficient σs(λ)
radiative transfer equation
phase function p(θ, λ)
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
Chandrasekhar 1960, Ishimaru 1978

4. Parameters are important
extinction coefficient σt(λ)
scattering coefficient σs(λ)
phase function p(θ, λ)
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

5. Parameters are non-trivial≈ ≠
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

6. Light transport is complicated
rendering:
Monte-Carlo techniques
accurate (exact at convergence)
efficient (relatively)
rendering
(σt σs p(θ))
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
Veach 1997

7. Light transport is complicated
inverse rendering:
fancy term for image-based parameter estimation
no standard way to solve it
rendering
(σt σs p(θ))
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
inverse rendering
Bal 2009

8. Inverse rendering approximations
single-scattering approximation
[Narashimhan et al. 2006]
[Mukaigawa et al. 2010]
thin materials, diluted liquids
only approximate isolation
(σt σs p(θ))
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
inverse rendering

9. Inverse rendering approximations
diffusion approximation
[Jensen et al. 2001]
[Papas et al. 2013] … … … …
optically thick materials
parameter ambiguity
(σt σs p(θ))
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
inverse rendering
Wyman et al. 1989, Jensen et al. 2001, Papas et al. 2013

10. Inverse rendering approximations
parameterization of phase function
[Chen et al. 2006]
Henyey-Greenstein lobes
[Donner et al. 2008]
[Fuchs et al. 2007]
[Goesele et al. 2004]
[Gu et al. 2008]
single-parameter family:
[Hawkins et al. 2005]
[Holroyd et al. 2011]
g∈ −1,1
[McCormick et al. 1981]
not general enough
[Pine et al. 1990]
The most commonly used phase function in computer graphics is the Henyey-Greenstein family. These are parabola-like lobes, controlled by a single parameter g. For different values of this parameter, we can have backward scattering, isotropic, and forward scattering lobes.
Thinking of the phase function as a spherical distribution, the parameter g is equal to the first moment mu1, also commonly referred to as the average cosine
[Prahl et al. 1993]
[Gkioulekas et al. 2013]
[Wang et al. 2008]

11. Our contributions dictionary-based material representation
operator-theoretic framework for inverse volume rendering without approximations
𝜕𝐸 𝜕 𝑤 𝑞 =2 𝑤 𝑞 𝑆 𝐼−𝐾 −1 𝐿 𝑖 −𝜇
𝑆 𝐼−𝐾 −1 𝐾 𝑞 𝐼−𝐾 −1 𝐿 𝑖
physical setup implementation and accurate measurement of broader classes of materials
The most commonly used phase function in computer graphics is the Henyey-Greenstein family. These are parabola-like lobes, controlled by a single parameter g. For different values of this parameter, we can have backward scattering, isotropic, and forward scattering lobes.
Thinking of the phase function as a spherical distribution, the parameter g is equal to the first moment mu1, also commonly referred to as the average cosine

12. Basic idea appearance matching collect images μ1 μ2 μ3 μ4 μ5 μ6 μ7 μ8μ9
find material that matches them
m = (σt σs p(θ))
min Σ ǁ μs - render(m, scenes) ǁ2
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions. m

13. Dictionary parameterization
dictionary of phase functions p9 p7
p10
p11 p6 p8 p5 p1 p2 p3 p4
D = {p1, p2, …, pQ}
express arbitrary phase function in terms of dictionary p
p = Σ wq pq
similarly for all parameters
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

14. Dictionary parameterization
dictionary of materials
D = {m1, m2, …, mQ}
express arbitrary material in terms of dictionary
m = Σ wq mq
similarly for all parameters
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

15. Basic idea appearance matching collect images μ1 μ2 μ3 μ4 μ5 μ6 μ7 μ8μ9
find material that matches them
m = (σt σs p(θ))
m = (D, w)
min Σ ǁ μs - render( m w
, scenes) ǁ2
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions. m w

16. Temporally resolved light transport
propagation step τ = c Δt τ τ τ τ
propagation probabilities:
phase function p(θ)
scattering coefficient σs
extinction coefficient σt
continue straight:
π1 = 1 - τ σt
absorption event:
π2 = τ (σt - σs)
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
volume event
scattering event:
π3 = τ σs (p(θ))

17. Temporally resolved light transport
linear algebra of single
propagation step τ = c Δt
Lt+Δt = K Lt
Lt+Δt =
(π1I + π3 P) Tτ Lt
single-step matrix K
straight:
function of material: m = (σt σs p(θ))
Lt+Δt,str = Tτ Lt
π1 = 1 - τ σt
absorption:
Lt+Δt,abs = 0
π2 = τ (σt - σs)
lightfield L
scattering:
Lt+Δt,sca = P Tτ Lt
π3 = τ σs
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

18. Temporally resolved light transport
linear algebra of multiple
L1 = K Li
propagation steps τ = c Δt
L2 = K2 Li
L3 = K3 Li
Lt+Δt = K Lt
L4 = K4 Li
single-step matrix K
L5 = K5 Li
L = Σ Kn Li = (I - K)-1 Li
L6 = K6 Li
lightfield L
t = 4 Δt
t = 6 Δt
t = 2 Δt
t = 7 Δt
t = 5 Δt
t = 3 Δt
t = 8 Δt
t = 9 Δt
t = Δt
t = 0
rendering matrix R
L7 = K7 Li
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
L8 = K8 Li

19. Dictionary parameterization
dictionary of materials
D = {m1, m2, …, mQ}
express arbitrary material in terms of dictionary
m = Σ wq mq
single-step matrix K becomes
K = Σ wq Kq
rendering matrix R becomes
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
R = (I - Σ wq Kq)-1

20. Basic idea appearance matching collect images μ1 μ2 μ3 μ4 μ5 μ6 μ7 μ8μ9
find material that matches them
m = (D, w)
min Σ ǁ μs - Ss (I - Σ wq Kq)-1 Li,s) ǁ2
min Σ ǁ μs - render( w
, scenes) ǁ2
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions. w

21. Optimization appearance matching loss function
E(w) = Σ ǁ μs - Ss (I - Σ wq Kq)-1 Li,s) ǁ2
differentiable!
𝜕E w 𝜕 w q = 2wq Σ(Ss (I - Σ wq Kq)-1 Li,s - μs)
Ss (I - Σ wq Kq)-1 Kq (I - Σ wq Kq)-1 Li,s
after some simplification
𝜕E w 𝜕 w q = 2wq Σ(Ss R(w) Li,s - μs) Ss R(w) Kq R(w) Li,s
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
use your favorite gradient descent variant to do inverse rendering

22. The world is continuous
linear algebra of temporally resolved light transport
Lt+Δt = K Lt
single-step matrix K
L = R Li = (I - K)-1 Li
rendering matrix R
lightfield L
light transport can be described using matrix-vector products
(continuous function)
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

23. The world is continuous
operator theory of temporally resolved light transport
Lt+Δt = K Lt
single-step operator K
L = R Li = (I - K)-1 Li
rendering operator R
lightfield L
matrix-vector products become (Monte-Carlo) rendering
(continuous function)
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
Arvo 1995

24. Optimization (operator version)
operator version of gradient is identical
𝜕E w 𝜕 w q = 2wq Σ(Ss R(w) Li,s - μs) Ss R(w) Kq R(w) Li,s
three cascaded Monte-Carlo operations
noisy estimates use stochastic gradient descent
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
R(w) Kq
R(w)

25. Basic idea appearance matching collect images μ1 μ2 μ3 μ4 μ5 μ6 μ7 μ8μ9
find material that matches them
m = (D, w)
min Σ ǁ μs - Ss (I - Σ wq Kq)-1 Li,s) ǁ2
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions. w

26. Acquisition setup front lighting material sample backlighting
hyperspectral camera
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

27. Acquisition setup front lighting material sample backlighting
hyperspectral camera
top rotation stage
bottom rotation stage
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

28. Acquisition setup frontlighting material sample front lighting
backlighting
hyperspectral
camera
hyperspectral camera
top rotation
stage
top rotation stage
bottom rotation
stage
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

29. Acquisition setup MEMS light switch blue (480 nm) laser
green (535 nm) laser
red (635 nm) laser
RGB combiner
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.
light generation

30. Acquisition setup Some acquisition statistics:
3 lighting angles (for both front- and back-lighting)
3 viewing angles
LDR images at 18 exposures
3 wavelengths
total 54 HDR measurements per material
75 minutes capture per material
Translucent appearance is caused by scattering of light inside material volumes. This process is described by the radiative transfer equation, and is controlled by three material-dependent parameters. Let’s take a closer look: As light travels through a medium, the extinction coefficient controls the distance before a volume events occurs. Then, at each volume event, light may get absorbed with some probability determined by the absorption coefficient. Otherwise, light is scattered, meaning that travels in a different direction, as determined by the phase function. In general these parameters are functions of wavelength, but we will be ignoring this dependence dealing only with “grayscale” materials. We will be focusing on the phase function in this talk.
The phase function is a probability distribution on the sphere of directions, and is often assumed to be spherically symmetric. We will also use this assumption, and therefore use polar plots to represent them: For light incoming from the left, the plot shows the probability that light will get scattered at each outgoing direction. We show for reference an isotropic phase function, meaning one that scatters equally in all directions.

31. Validation materials reference nanodispersions
particles of known molecular type dispersed in liquid media
very precise size distributions (NIST Traceable Standards)
ground-truth materials: parameters predicted exactly by Mie theory
polystyrene
monodispersions
aluminum oxide
polydispersions
Frisvad et al. 2007

32. Validation materials comparison of predicted and measured parameters
polystyrene 1
polystyrene 2
polystyrene 3
aluminum oxide
all parameters estimated within 4% error

33. Validation materials
comparison of captured and rendered images in novel geometries
captured
rendered
rendered with HG
profiles
polystyrene 1
images under unseen geometries predicted within 5% RMS error

34. Validation materials
comparison of captured and rendered images in novel geometries
captured
rendered
rendered with HG
profiles
polystyrene 2
images under unseen geometries predicted within 5% RMS error

35. Validation materials
comparison of captured and rendered images in novel geometries
captured
rendered
rendered with HG
profiles
polystyrene 3
images under unseen geometries predicted within 5% RMS error

36. Validation materials
comparison of captured and rendered images in novel geometries
captured
rendered
rendered with HG
profiles
aluminum oxide
images under unseen geometries predicted within 5% RMS error

37. Measured materials highly scattering liquids highly absorbing liquids
hand cream
curacao
highly scattering liquids
olive oil
shampoo
robitussin
mixed soap
whole milk
highly absorbing liquids
solids
milk soap
wine
liquid clay
mustard
coffee
reduced milk

38. Measured materials whole milk reduced milk mustard shampoo hand cream
liquid clay
milk soap
mixed soap
glycerine soap
robitussin
coffee
olive oil
curacao
wine

39. Measured materials

40. Measured materials

41. Measured materials

42. Discussion
more interesting materials: heterogeneous volumes, polarization-dependent behavior, birefringent and fluorescing materials
use of our optimization framework with other setups: alternative lighting (basis, adaptive, high-frequency), geometries, or imaging (transient imaging)
simpler optimization and setup: trade-offs between accuracy, generality, mobility, and usability

43. Take-home messages dictionary-based material representation
operator-theoretic framework for inverse volume rendering without approximations
𝜕𝐸 𝜕 𝑤 𝑞 =2 𝑤 𝑞 𝑆 𝐼−𝐾 −1 𝐿 𝑖 −𝜇
𝑆 𝐼−𝐾 −1 𝐾 𝑞 𝐼−𝐾 −1 𝐿 𝑖
physical setup implementation and accurate measurement of broader classes of materials
The most commonly used phase function in computer graphics is the Henyey-Greenstein family. These are parabola-like lobes, controlled by a single parameter g. For different values of this parameter, we can have backward scattering, isotropic, and forward scattering lobes.
Thinking of the phase function as a spherical distribution, the parameter g is equal to the first moment mu1, also commonly referred to as the average cosine

44. Acknowledgments
Shuang Zhao
Todd Zickler
Kavita Bala
Anat Levin

45. Thank you for your attention