1. Rendering falling snow using an inverse Fourier transform
Michael Langer Linqiao Zhang
School of Computer Science McGill University
[Show title slide as you are being introduced. Everyone will read it. You should NOT repeat what the title is and say who you are. The person who introduces you will do this.]

2. Falling snow
Today I’m going to tell you about a method that we have been investigating for rendering falling snow. If you have seen falling snow, then you know that it produces a very beautiful visual effect. We are trying to understand how to generate this visual effect. [PAUSE]
Let’s take a simple cartoon example illustrated here. Suppose that each snowflake is falling at the same 3D velocity. One can make two observations. First, because of perspective, snowflakes that are closer to the eye move faster in the image than snow flakes that are further from the eye. This is commonly known as motion parallax. Motion parallax arises because snowflakes that are closer to the eye cover a greater visual angle per second than snowflakes that are far from the eye [PAUSE.] A second observation, which also is due to linear perspective, is that snowflakes that are farther away will be smaller in the image. We will use these two observations in our method, and I will return to the two observations in about 5 minutes from now. [PAUSE] In the meantime, let me give you some of the background of the work.

3. If someone gives you the task of rendering falling snow, the first thing you would do is to model each of the snowflakes and make them fall with a certain 3D velocity. Here we show a very simple video, rendered using this technique. The “snowflakes” are spheres, randomly placed in a view volume. The spheres are all moving at the same 3D velocity. As we observed before, spheres that are closer to the camera are moving faster than the spheres that are far away and snowflakes that are nearer to the camera are bigger. [PAUSE] This video is obviously not very realistic. So you might then try to improve the rendering by modelling the shape of the snowflakes, or the distribution of snowflakes in space, ortheir trajectories by taking account of friction and turbulence and wind, and even the optics of light reflecting off the snowflakes. People have done these sorts of things. The method in general is known as “particle systems”.

4. Particle systems method for rendering falling snow
W. Reeves, ACM Trans. Graphics ’83
K. Sims, SIGGRAPH ‘90
M. Shinya and A. Fournier, Eurographics ’92
T. Nishita et al. Eurographics ’97
P. Fearing, SIGGRAPH ‘00
The concept of particle systems as a general rendering method was introduced by Bill Reeves in the early 1980s. Particle systems have been used for many applications in computer graphics, not just falling snow. [PAUSE] If one just consider the problem of rendering snow, then one can find many particle system methods. Some of these are listed here. The method listed here include both the problem of rendering falling snow, and rendering fallen snow, that is, snow that is falling, and snow that has already fallen onto a surface. We will focus on the falling snow problem. [PAUSE] There is no doubt that one can make beautiful realistic videos of falling snow using particle systems But particle systems are not the only way to render falling snow. Today I will tell you about another method for rendering falling snow which we have been developing.

5. Rendering falling snow using an inverse Fourier transform
The method that I will describe is to render falling snow using inverse Fourier transform, not using particle systems. [PAUSE]

6. “Spectral Synthesis” noise, turbulence (Perlin, ’85)
clouds (Gardner,’84, ’85)
terrains (Voss, ’85)
ocean waves (Mastin et al, ’87)
wind, fluids and smoke (Stam and Fiume, ’93, Nishita and Fournier ‘92)
The general idea of rendering images using an inverse Fourier transform has been around in computer graphics for many years. The method is
often called spectral synthesis. The idea first appeared in computer graphics in the mid-1980s. Basically, you synthesis some function as a sum of sinusoidal images. The amount of each sinusoid is a function of wavelength, which is why you refer to a “spectrum”. For example, Perlin introduced his NOISE and TURBULENCE functions in Others have rendered clouds, fractal mountain terrains, ocean waves, as well as wind, fluids and smoke using this method. I have listed just a small number of examples here.

7. “Spectral Synthesis” method
Model the power spectrum
Compute the inverse Fourier transform
Render
The spectral synthesis method has three steps. First, you model the spectrum of the physical phenomenon you are interested in, whether it is a terrain shape, or a cloud, or whatever. To do this, you need to learn about the physics of the phenomenon you are trying to render. The spectral model gives the relative strengths of the various sinusoidal components. Second, you need to compute the inverse Fourier transform so that you get a signal in the spatial domain(or the space-time domain (depending on what you are modelling). Computing an inverse Fourier transform is straightforward. The third step is to render the function that you have synthesized. For example, in the case of a terrain, you need to choose the lighting and surface material. [PAUSE]
For the case of falling snow, we first need to model the power spectrum of an image of falling snow. Second, we need to compute the inverse Fouriert ransform (this is straightforward). Third, we need to render the snow. The two contributions of the research I am presenting are steps 1 and 3. I will next derive a model of the power spectrum of falling snow. Then we will show how to render it. [PAUSE] Let’s begin with the first step, which is to model the power spectrum of an image of falling snow.

8. Fourier transform of motion
For those of you who are not familiar with Fourier analysis of image motion, let me just give the basic idea. Any video can be written as the sum of a set of moving sine waves. Each of these sine waves is characterized by a spatial frequency in the horizontal direction (the x direction), a spatial frequency in the vertical direction (the y direction) and a temporal frequency. The moving sine wave shown here has 8 cycles across the image in both the x and y directions. And it has a certain temporal frequency as well. The temporal frequency depends on the speed of the moving sine wave. The intensity at each point varies sinusoidally over time. The number of cycles per second is the temporal frequency. So we have three frequencies: a spatial frequency in x, a spatial frequency in y, and a temporal frequency. The basic idea of the Fourier transform is to write an video as a sum of such moving sine waves.
Any video can be written as the
sum of moving sine waves.

9. Fourier transform of motion
f , f are spatial frequencies
f is temporal frequency
(v , v ) is image velocity
x y t
x y
If we look at one of these moving sine wave components. Then we observe a very simple relationship between the spatial and temporal frequencies and the velocity of the moving sine wave. Let fx, fy be the spatial frequencies. Let ft be the temporal frequency. And let
(vx,vy) be the velocity. Then all there variables are related by the equation shown here. [PAUSE]
v f + v f + f = 0
x x y y t

10. Two observations about falling snow
Now let’s return to the two observations about falling snow that we made earlier. The first observation was that far snowflakes are smaller than big snowflakes. The second observation was that far snowflakes are slower than big snowflakes. [PAUSE] We are going to now use these observations to derive a model for spectral synthesis of falling snow.
Far snowflakes are smaller.
Far snowflakes are slower.

11. Observation 1
Far snowflakes correspond to higher spatial frequencies.
distance d =
f f
x y
Lets take the observation that far snowflakes are smaller. In the frequency domain, this observation implies certain things about how energy is distributed over spatial frequencies. If far snowflakes are smaller in the image, then far snowflakes contribute to higher spatial frequencies, that is, far snowflakes contribute to higher f values. So, if d is the distance to a snowflake, then d goes like spatial frequency.

12. Observation 2 Fall snowflakes are slower. v ~ 1 d
The second observation is that far snowflakes move slower in the image. So, again if d is distance, then the image velocity of a given snowflake goes like 1 over the distance d.

13. Observations 1& 2 f + f Thus, v ~ 1 2 2 x y
f f
x y
If we put these two observations together, we get that the velocity depends on the spatial frequency. High spatial frequencies should move with slower image velocities. [PAUSE] This result is not obvious, so I’ll say it again. For falling snow, high spatial frequencies move with slower image velocities. [PAUSE.]
We are almost finished with our model. The last step goes as follows.

14. Observation 3 f f For vertically falling snow v = t y y
Recall earlier in the talk that when we have a sine wave moving with a certain image velocity, then there is a linear relation between spatial and temporal frequencies. For the case of vertically falling snow, the x component of the velocity is 0, and so the relationship reduces to the equation shown here. Vertical speed is the ratio of temporal frequency to spatial frequency. [PAUSE] Let’s call this observation 3. y

15. Observations 1, 2, 3 imply : f f f + f =
“dispersion relation” for motion parallax f y f t
f f
x y
So now we put all three observations together. We can cancel out the velocity variable and we get that the temporal frequency is a non-linear function of the spatial frequencies. This equation here is for the case of vertically falling snow. A slightly more complicated equation is needed for snow falling in any image direction. [PAUSE] Such a relation between temporal and spatial frequencies is sometimes called a dispersion relation by physicists. So what we have derived loosely here is a dispersion relation for motion parallax, in particular, for falling snow.

16. “Spectral Synthesis” method
Model the power spectrum
Restrict the spectrum to the dispersion surface in (f , f , f )
set 1/f amplitude and random phase over upper 3 octaves of spatial frequencies
Compute the inverse Fourier transform
Render
x y z
So, lets return to the basic idea of the spectral synthesis method. We are still at the first step. We model the power spectrum by putting energy only on those travelling sine wave components that satisfy the dispersion relation shown on the previous slide. That is, any spatio temporal frequency combinations that do NOT fall on the dispersion surface will have zero energy and will not contribute. [PAUSE] We also need to specify how much energy (amplitude) to put at each frequency combination on the dispersion surface. For this, we use 1/f noise. This has the effect of putting the constant amount of energy in each constant octave band of frequencies. We only put energy into high spatial frequencies, namely 3 octaves of high spatial frequencies. The reason is that snowflakes tend to be small and thus contribute mostly to higher spatial frequencies. [PAUSE] Ok, so that is how we model the Fourier transform of the image of falling snow. The second step is then to compute the inverse Fourier transform. This gives us an image sequence. The following slide shows an example.

17. Inverse Fourier transform of dispersion surface
[Show this slide and let people look at it for at least 30 seconds. Comment that we have used diagonal motion here, rather than vertical motion, to demonstrate that the technique does not just apply to vertical motion. Also make sure that you say the main point : bigger stuff moves faster. ]
Bigger stuff moves faster.

18. “Spectral Synthesis” method
Model the power spectrum
Compute the inverse Fourier transform
Render
Treat synthetic snow as a time-varying opacity function a(x, y, t)
Composite the snow with a background image or video

19. Composite with background image
I(x,y,t) = a(x, y, t) I
+ (1 - a(x, y, t)) I (x,y)
amb bg
To render the snow, we treat the snow as if it has one intensity value only, namely I_amb. This is an intensity value close to Since snow is white the intensity of the snow should be roughly the same as the intensity of ambient light in the scene. Obviously this is a crude but simple approximation. The function alpha is the synthesized opacity of the snow. We also need a background image of a scene, if we want to make a video of the snow falling in a scene.

20. Background image
Jimmy’s place, painting by Gary Johnson

21. Composite snow on top of background image