Convolution (FFT) Bloom David Hill Principal Mathematician Epic Games 5/25/2017

Bloom: Why Invest? A convincing bloom adds subtle but impactful physically motivated effect. Currently feasible on today’s high end hardware.. Adapt film / offline approach for use in real time Additional realism with highly detailed effects Provide ability to customize the camera / eye response Target Cinematics and High End hardware

Bloom As Weak Scattering Weak Scattering: Some Causes Haze Car Windshield (mine is always a little dirty) Eyelash Diffraction Inner-Camera Effects Fraunhofer Diffraction Physically-Based Glare Effects for Digital Images (1995, Spencer et al) Glare Generation Based on Wave Optics (2004, Kakimoto et al, Computer Graphics Forum)

Bloom As Weak Scattering A single pin point remains mostly focused Model Assumptions: Every point of light is scattered the same For most sources the scattered amount is imperceptible The scattering from bright sources creates “bloom” In Camera scattering of a point

Bloom: Standard Game-Style Weak Scattering Sum of Gaussian Filter Image down-sampled multiple times and Gaussian Blur applied Resulting images are summed and added to original Every pixel scatters light in a symmetric way to its neighbors Very Fast! But... Standard Bloom in UE4: Recommended for game use

Bloom As Weak Scattering The camera response to a single bright source: Close Up of center Center pixel about 10,000 times brighter than any other point in this .exr Camera scattering of point Appears to be a very small star Full image, adjusted contrast Detail extents far across image!

Bloom Scatter As Gather: Convolution View this as a weighted sum, here in one dimension : Original Value Filtered Value Filter Each Pixel Scatters to all the others according to the filter. For Example, simple smoothing

Bloom Scatter As Gather: Convolution Every point is weighted sum of all other points N points gather from all N points. O(N^2) We need a faster way! -> FFT

Bloom Scatter As Gather: Convolution Convolution With Fast Fourier Transform: Image_Frequencies = FFT(Image) Filter_Frequencies = FFT(Filter) Convolved_Frequencies = Image_Frequencies x Filter_Freqencies Convolved_Image = InverseFFT(Convolved_Frequencies)

Bloom: Why FFT Convolution? Better Scaling FFT from signal to frequencies - O(N Log N) Convolution in frequency space - O(N) FFT from frequencies to signal - O(N Log N) Overall Scaling O(N Log N) For our purposes, the Fast Fourier Transform (FFT) is an acceleration technique for convolution.

Bloom: What is Fourier Transform? Any finite length signal can be expresses as a sum of sines and cosines Signal Amplitude of each frequency Sine & Cos Inverse computes amplitudes of each frequency Low-Pass filter: only used first few V_k

Bloom: Fast Fourier Transform? Sines and Cosines have symmetries that can be exploited speeding the transform Split the sum into even and odd terms, and symmetry saves work! Now recurse, each small transform (Even, Odd) and be treated as new transforms And subdivided. Due to this splitting FFT does best with power_2 signals.

Bloom : Parallel FFT for GPU GPU: Highly parallel, but with thread-communication limitations. Similar restriction are found in supercomputing architectures. Stockham Formulation: Parallel formulation Used for Vector Computers (1987 D Baily, Journal of Supercomputer Applications) Applied to GPUs (2008 Govindaraju, Proceedings of ACM/IEEE on Supercomputing) Our implementation largely follow this Each Scanline Transformed independently Assigned to a thread group with group shared memory

Bloom: Compute Shaders FFT implemented as compute shaders Pass Forward Horizontal Transform Uses 2-for-1 trick to transform (r,g) and (b,a) Forward Vertical Transform Convolution, i.e. Multiply, with cached pre-transformed Kernel Inverse Vertical Transform Inverse Horizontal Transform Inverts 2-for-1 trick From GPUVisualizer