Signal Processing and Representation Theory Lecture 4

Outline: Review Fourier Transforms Applications Invariant Descriptors

Representation Theory Review Circle: If G is the group of 2D rotations / reflections, acting on the space of functions on the circle, the irreducible sub-representations of G are the 1D subspaces spanned by the complex exponentials: V k =Span{e ik  }

Representation Theory Review Circle: Given a function f defined on the circle, we can obtain a rotation / reflection invariant representation by computing the Fourier decomposition: and storing the norms of the Fourier coefficients:

Representation Theory Review Circle: Given two functions f and h defined on the circle, we can correlate the functions by computing the Fourier decompositions: multiplying the Fourier coefficients: and computing the inverse Fourier transform.

Representation Theory Review Sphere: If G is the group of 3D rotations / reflections, acting on the space of functions on the sphere, the irreducible sub-representations of G are the (2d+1)- dimensional subspaces spanned by the spherical harmonics of frequency d:

Representation Theory Review Sphere: Given a function f defined on the sphere, we can obtain a rotation / reflection invariant representation by computing the spherical harmonic decomposition: and storing the norms of the frequency components:

Representation Theory Review Sphere: Given two functions f and h defined on the sphere, we can correlate the functions by computing the spherical harmonic decompositions: multiplying the spherical harmonic coefficients: and computing the inverse Wigner-D transform.

Outline: Review Fourier Transforms Applications Invariant Descriptors

Representation Theory Fourier Transforms 2D: If we have the space of functions in the plane, we can consider the representation obtained by the group of translations. Since translations are commutative, Schur’s Lemma tells us that the irreducible representations are all one-dimensional.

Representation Theory Fourier Transforms 2D: The irreducible representations are the sub-spaces spanned by the functions: Translating each function by (x 0,y 0 ) we get:

Representation Theory Fourier Transforms 2D (Invariance): If f(x,y) is a function defined on the plane, we can express the function in terms of its Fourier decomposition: and obtain a rotation invariant representation by storing the energy in each frequency:

Representation Theory Fourier Transforms 2D (Correlation): If f and g are functions defined on the plane whose Fourier decompositions are: the correlation of f with g over the space of translations can be computed by multiplying the Fourier coefficients:

Representation Theory Fourier Transforms 3D: If we have the space of functions in 3D, we can consider the representation obtained by the group of translations. Since translations are commutative, Schur’s Lemma tells us that the irreducible representations are all one-dimensional.

Representation Theory Fourier Transforms 3D: The irreducible representations are the sub-spaces spanned by the functions: Translating each function by (x 0,y 0,z 0 ) we get:

Representation Theory Fourier Transforms 3D (Invariance): If f(x,y,z) is a function defined in 3D, we can express the function in terms of its Fourier decomposition: and obtain a rotation invariant representation by storing the energy in each frequency:

Representation Theory Fourier Transforms 3D (Correlation): If f and g are functions defined in 3D whose Fourier decompositions are: the correlation of f with g over the space of translations can be computed by multiplying the Fourier coefficients:

Outline: Review Fourier Transforms Applications –Circle –2D –Sphere –3D Invariant Descriptors

Representation Theory Applications Circle: If we have a real-valued function f on the circle, we can express the function in terms of its Fourier decomposition: where a k,b k  ℝ and b k =-b -k.

Representation Theory Applications Circle: Given the space of real-valued functions on the circle, the Fourier decomposition can be used for correlation and invariants-extraction with respect to the group of 2D rotations / reflections.

Representation Theory Applications Circle: What if we consider the smaller group of axial flips: that arise due to the ambiguity in PCA alignment?

Representation Theory Applications Circle: Initial f()f() y-flip f(π-)f(π-) x-flip f(-  ) x,y-flip f(π+)f(π+)

Representation Theory Applications Circle: Because axial flips are a subgroup of the rotations / reflections, sub-representations for the entire group of rotations / reflections are also sub-representations for the group of axial flips.

Representation Theory Applications Circle: Initial f()f() y-flip f(π-)f(π-) x-flip f(-  ) x,y-flip f(π+)f(π+)

Representation Theory Applications Circle: Initial f()f() y-flip f(π-)f(π-) x-flip f(-  ) x,y-flip f(π+)f(π+)

Representation Theory Applications Circle: Initial f()f() y-flip f(π-)f(π-) x-flip f(-  ) x,y-flip f(π+)f(π+)

Representation Theory Applications Circle: Initial f()f() y-flip f(π-)f(π-) x-flip f(-  ) x,y-flip f(π+)f(π+)

Representation Theory Applications Circle: Initial f()f() y-flip f(π-)f(π-) x-flip f(-  ) x,y-flip f(π+)f(π+)

Representation Theory Applications Circle: If f is a real-valued function on the circle expressed in terms of its Fourier decomposition as: an axial-flip invariant representation can be obtained by storing the norms of the real and imaginary components:

Representation Theory Applications Circle: If f and h are real-valued functions on the circle, we could compute the correlation of f with h over all axial flips by comparing at each axial flip independently. This would take four times as long.

Representation Theory Applications Circle: Instead, if we express f and h in terms of their Fourier decomposition: then the correlation of f with h becomes:

Representation Theory Applications Circle: By computing the different summations independently: we can compute the correlation at the different axial flips more efficiently. α (real, even) β (imaginary, even) γ (real, odd) δ (imaginary, odd)

Representation Theory Applications Circle: α (real, even) β (imaginary, even) γ (real, odd) δ (imaginary,odd) Initialαβγδ x- flipα-β-βγ-δ-δ y- flipα-β-β-γ-γδ x,y-flipαβ-γ-γ-δ-δ Initialy-flipx-flipx,y-flip

Representation Theory Applications Circle: So by pre-computing the values of α, β, γ, and δ, we can compute the correlation at each of the four axial flips by summing the values of α, β, γ, and δ with the appropriate sign. Instead of taking 4 times as long, it takes 16 extra arithmetic ops. to compute the correlation values. Initialx-flipy-flipx,y-flip Correlationα+β+γ+δα+β+γ+δα-β+γ-δα-β+γ-δα-β-γ+δα-β-γ+δα+β-γ-δα+β-γ-δ

Representation Theory Applications 2D (Rotation): If we are given a function f defined on the set of points inside the unit disk (x 2 +y 2  1), we can express the function in terms of radius and angle:  r

Representation Theory Applications 2D (Rotation): If we hold the radius fixed, we get a function defined on a circle: and we can apply methods from functions on a circle to obtain rotation invariants and to correlate.  r

Representation Theory Applications 2D (Rotation): To get rotation invariants, we can express the initial function f as a collection of circular functions, obtained by restricting f to different radii:  r

Representation Theory Applications 2D (Rotation): Computing the Fourier decomposition of each circular restriction: we can obtain a rotation invariant representation by storing the norms of the different frequency components of the different circular restrictions:

Representation Theory Applications 2D (Rotation): To correlate two functions f and h, we can express the initial functions as collections of circular functions, obtained by restricting to different radii:

Representation Theory Applications 2D (Rotation): Then the correlation can be obtained by multiplying the Fourier coefficients of each of the restrictions: Complexity: 1.2N forward Fourier Transforms: O(N 2 logN) 2.Frequency multiplication:O(N 2 ) 3.One inverse Fourier Transform:O(N logN)

Representation Theory Applications 2D (Axial Flips): Given two functions f and h defined on the plane, we can express the two functions in terms of the Fourier decomposition of their radial restrictions: with a r,k,b r,k,c r,k,d r,k  ℝ.

Representation Theory Applications 2D (Axial Flips): By computing the different summations independently: the correlation at all four axial flips can be computed with only 16 extra arithmetic operations. α (real, even) β (imaginary, even) γ (real, odd) δ (imaginary, odd)

Representation Theory Applications Sphere: If we have a real-valued function f on the sphere, we can express the function in terms of its spherical harmonic decomposition: where a l m,b l m  ℝ, a l -m =(-1) m a l m, and b l -m =(-1) m b l m.

Representation Theory Applications Sphere: Given the space of real-valued functions on the sphere, the spherical harmonic decomposition can be used for correlation and invariants-extraction with respect to the group of 3D rotations / reflections.

Representation Theory Applications Sphere: What if we consider the smaller group of axial flips: that arise due to the ambiguity in PCA alignment?

Representation Theory Applications Sphere: Because axial flips are a subgroup of the rotations / reflections, sub-representations for the entire group of rotations / reflections are also sub-representations for the group of axial flips.

Representation Theory Applications Sphere: Using the facts that: and the harmonic of frequency l are even (resp. odd) when l is even (resp. odd), we get:

Representation Theory Applications Sphere: An axial flip invariant representation can be obtained by computing the spherical harmonic decomposition: and separately storing the norms of the real and imaginary components of the harmonic coefficients:

Representation Theory Applications Sphere: and in a similar manner as before, we can compute the correlation at all eight axial flips with only 64 extra arithmetic operations.

Representation Theory Applications 3D (Rotation): If we are given a function f defined on the set of points inside the unit disk (x 2 +y 2 +z 2  1), we can express the function in terms of radius and angle:

Representation Theory Applications 3D (Rotation): If we hold the radius fixed, we get a function defined on a sphere: and we can apply methods from functions on a sphere to obtain rotation invariants and to correlate.

Representation Theory Applications 3D (Rotation): To get rotation invariants, we can express the initial function f as a collection of spherical functions, obtained by restricting f to different radii:

Representation Theory Applications 3D (Rotation): Computing the spherical harmonic decomposition of each spherical restriction: we can obtain a rotation invariant representation by storing the norms of the different frequency components of the different spherical restrictions.

Representation Theory Applications 3D (Rotation): To correlate two functions f and h, we can express the initial functions as a collection of spherical functions, obtained by restricting to different radii:

Representation Theory Applications 3D (Rotation): Then the correlation can be obtained by multiplying the Fourier coefficients of each of the restrictions: Complexity: 1.2N forward spherical harmonic transforms: O(N 3 log 2 N) 2.N intra-frequency multiplication: O(N 4 ) 3.One inverse Wigner-D transform: O(N 4 )

Representation Theory Applications 3D (Axial Flips): Given two functions f and h defined in 3D, we can express the two functions in terms of the spherical harmonic decomposition of their radial restrictions and obtain a method that computes the correlation at each axial flip with only 64 extra arithmetic operations.

Representation Theory Applications Summary: TranslationRotationAxial Flip Circle 1D Fourier TransformReal/Imaginary Even/Odd Fourier Transform 2D 2D Fourier TransformCircular Restrictions 1D Fourier Transform Circular Restrictions Real/Imaginary Even/Odd Fourier Transform Sphere Spherical Harmonics Wigner-D Transform Real/Imaginary Even/Odd Spherical Harmonic Transform 3D 3D Fourier TransformSpherical Restrictions Spherical Harmonics Wigner-D Transform Spherical Restrictions Real/Imaginary Even/Odd Spherical Harmonic Transform

Outline: Review Fourier Transforms Applications Invariant Descriptors –Shape Histograms (Shells) –Shape Distributions (D2) –Extended Gaussian Images (EGI)?

Representation Theory Shape Histograms (Shells) Obtain a rotation invariant by storing the amount of the shape that resides within each spherical shell. ModelShape Histogram (Shells)

Representation Theory Shape Histograms (Shells) This amounts to storing the constant order component of a spherical restriction. We could get a more discriminating descriptor by storing other frequency information.

Representation Theory D2 Distributions Obtain a translation and rotation invariant by storing the distribution of distances between pairs of points on the model. D2 Distribution 3D Model p q Distance Distribution

Representation Theory D2 Distributions We can decompose the computation of the D2 descriptor into two steps: –Voting on displacement –Computing the average over spherical restrictions

Representation Theory D2 Distributions Step 1: For each pair (p,q) of surface points –Vote on bin corresponding to the vector p-q 3D Model p q

Representation Theory D2 Distributions Step 2: Compute the distance distribution by counting up the number of points in each spherical restriction. D2 Distribution Distance Distribution

Representation Theory D2 Distributions Rotation Invariance: Obtained by storing the amount of information in the constant order component. More rotation invariant information can be obtained by storing the norms of the non-constant frequencies as well.

Representation Theory D2 Distributions Translation Invariance: The value in a bin can be obtained by translating the model and counting the amount of overlap between the original model and the translated one.

Representation Theory D2 Distributions Translation Invariance: Set f(x,y,z) to be the function that is equal to 1 at points on the boundary and 0 everywhere else. Then the function: is equal to 1 if and only if (x,y,z) and (x-x 0,y-y 0,z-z 0 ) are both on the boundary, otherwise the value is 0.

Representation Theory D2 Distributions Translation Invariance: For a fixed offset (x 0,y 0,z 0 ), the number of points which satisfy the property that both the point (x,y,z) and the point (x-x 0,y-y 0,z-z 0 ) are on the boundary is equal to the number of points at which: is equal to 1.

Representation Theory D2 Distributions Translation Invariance: Thus the number of points (x,y,z) which satisfy the property that both (x,y,z) and (x-x 0,y-y 0,z-z 0 ) are on the boundary is equal:

Representation Theory D2 Distributions Translation Invariance: If we write out f in terms of its frequency decomposition: the auto-correlation is:

Representation Theory D2 Distributions Translation Invariance: So translation invariance is obtained by representing the frequency components by their (square) norms.