1. Application to Natural Image Statistics
With V. de Silva, T. Ishkanov, A. Zomorodian

2. An image taken by black and white digital camera can be viewed as a vector, with one coordinate for each pixel
Each pixel has a “gray scale” value, can be thought of as a real number (in reality, takes one of 255 values)
Typical camera uses tens of thousands of pixels, so images lie in a very high dimensional space, call it pixel space, P

3. Lee-Mumford-Pedersen [LMP] study only high contrast patches.
Collection: 4.5 x 106 high contrast patches from a
collection of images obtained by van Hateren and van der Schaaf

4. Choose how to model your data
Lee-Mumford-Pedersen [LMP] study only high contrast patches.
Collection: 4.5 x 106 high contrast patches from a
collection of images obtained by van Hateren and van der Schaaf
Choose how to model your data

5. Choose how to model your data Consult previous methods.

6. Do what the experts do. Borrow ideas. Use what others have done.
What to do if you are overwhelmed by the number of possible ways to model your data (or if you have no ideas):
Do what the experts do.
Borrow ideas.
Use what others have done.

7. Carlsson et al used

8. embedded in the 7-dimensional sphere.
Carlsson et al used
The majority of high-contrast optical patches are concentrated around a 2-dimensional C1 submanifold
embedded in the 7-dimensional sphere.

9. Persistent Homology: Create the Rips complex
We can compute the number of clusters for a variety of diameters.
We start with 17 data points, so if the diameter is 0, we have 17 clusters. Increasing the diameter, these 2 balls intersect so I now have 16 clusters.
If we continue to increase the diameter, we will eventually create the complex we saw before with 5 clusters, etc until we only have one cluster left.
Eventually this entire page will be purple, but right now, we know have one component.
To choose the threshold, one can determine how long a particular number of clusters lasts, for example for what set of radii do we have five clusters. If we have five clusters for the largest set of radii, then have gives us a good idea where to set the threshold and which simplicial complex best models our data.
I have put links to better animations on my on my YouTube site which may better illustrate this persistence concept. Next month, we will also talk much more about persistence during the live lectures for this course. This is just a preliminary introduction.
0.) Start by adding 0-dimensional data points
is a point in S7

10. For each fixed e, create Rips complex from the data
is a point in S7
a one dimensional simplicial complex. Note that we have clustered our data into five disjoint connected sets. So this is one way to cluster our data – that is grouping our data points into disjoint sets based on some definition of similarity. In this case, we have 5 clusters.
We can now add higher dimensional simplices.
1.) Adding 1-dimensional edges (1-simplices)
Add an edge between data points that are close

11. For each fixed e, create Rips complex from the data
Thus we now have the Vietoris Rips simplicial complex. Note we get the same simplex by adding one dimension at a time
2.) Add all possible simplices of dimensional > 1.
is a point in S7

12. For each fixed e, create Rips complex from the data
In reality used Witness complex (see later slides).
Thus we now have the Vietoris Rips simplicial complex. Note we get the same simplex by adding one dimension at a time
2.) Add all possible simplices of dimensional > 1.
is a point in S7

13. Probe the data

14. Probe the data

15. Can use function on data to probe the data

16. Large values of k:
measuring density of large neighborhoods of x,
Smaller values mean we are using smaller neighborhoods
Large k = smoothed out version 

17. Eurographics Symposium on Point-Based Graphics (2004)
Topological estimation using witness complexes
Vin de Silva and Gunnar Carlsson

18. Eurographics Symposium on Point-Based Graphics (2004)
Topological estimation using witness complexes
Vin de Silva and Gunnar Carlsson

24. From: http://www.math.osu.edu/~fiedorowicz.1/math655/Klein2.html
Klein Bottle
From:

25. M(100, 10) U Q
where |Q| = 30
On the Local Behavior of Spaces of Natural Images, Gunnar Carlsson, Tigran Ishkhanov, Vin de Silva, Afra Zomorodian, International Journal of Computer Vision 2008, pp 1-12.

28. Combine your analysis with other tools

30.
Machine learning is a scientific discipline that explores the construction and study of algorithms that can learn from data.[1] Such algorithms operate by building a model from example inputs and using that to make predictions or decisions,[2]:2 rather than following strictly static program instructions.
Machine learning studies computer algorithms for learning to do stuff.
The emphasis of machine learning is on automatic methods. In other words, the goal is to devise learning algorithms that do the learning automatically without human intervention or assistance.

31. Image Categorization Training Testing Training Labels Training Images
Image Features
Classifier Training
Trained Classifier
Testing
Image Features
Trained Classifier
Prediction
Outdoor
Test Image

32. cs.brown.edu/courses/cs143/lectures/17.ppt

35. The Theory of Multidimensional Persistence, Gunnar Carlsson, Afra Zomorodian
"Persistence and Point Clouds" Functoriality, diagrams, difficulties in classifying diagrams,
multidimensional persistence, Gröbner bases, Gunnar Carlsson 

38. To install the TDA package on a PC:
   install.packages("TDA")
To install the TDA package on a Mac:
install.packages("TDA", type = "source")
XX = circleUnif(30)

39. Plot of data points Barcode
Each bar in the barcode represents a cycle in some Hi.
The red bar represents the element in H1 (i.e., the circle = 1 dimensional cycle = sum of edges where the boundary of this sum = 0).
Bars representing an element in H0 (i.e, 0-dimensional cycles = vertices) are drawn in black
Birth Death 

40. Plot of data points Barcode
Each bar in the barcode represents a cycle in some Hi.
A bar starts at the birth time of the cycle it represents and ends at its death time.
Birth Death 

41. Barcode
Birth Death 
A bar starts at the birth time of the cycle it represents and ends at its death time
For each cycle in Hi = bar in barcode, we can plot the point (birth, death)
where
birth = birth time of this cycle
death = death time of this cycle
Black point = cycle in H0.
Red triangle = cycle in H1.
(Birth, Death) 

42. Barcode This plot of points (birth, death) is called the
A bar starts at the birth time of the cycle it represents and ends at its death time
This plot of points
(birth, death)
is called the
Persistence Diagram
where we also throw in the diagonal.
(Birth, Death) 

43. H0 = < a, b, c, d : tc + td, tb + c, ta + tb>
H1 = <z1, z2 : t z2, t3z1 + t2z2 >
[ )
[ )
[ ) [
z1 = ad + cd + t(bc) + t(ab), z2 = ac + t2bc + t2ab

44. (3, 4) (1, 2) (2, 5) (0, 1) (0, ∞)  (0, 5) [ ) [ ) [ ) [
Since we can’t plot (0, ∞), we instead plot (0, 5) where
5 = maximum time = maximum threshold
= 3rd argument in ripsDiag(XX,maxdimension,maxscale, …)
[ )
[ )
[ ) [
(3, 4)
(1, 2)
(2, 5)
(0, 1)
(0, ∞)  (0, 5)

45. [ )
[ )
[ ) [
(3, 4)
(1, 2)
(2, 5)
(0, 1)
(0, 5)

46. [ )
[ )
[ ) [
(3, 4)
(1, 2)
(2, 5)
(0, 1)
(0, 5)
Remember to add the diagonal

47. [ )
[ )
[ ) [
(3, 4)
(1, 2)
(2, 5)
(0, 1)
(0, 5)
The diagonal will be useful when we compute distance between persistence diagrams
Remember to add the diagonal

48. The homology of a circle is as follows:
Rank of H0 = 1
since a circle has only one component
Rank of H1 = 1
since a circle has a single 1-d component
Rank of H2 = 0
since we don’t have any 2-d circles.

49. The homology of a circle is as follows:
Rank of H0 = 1
since a circle has only one component
Rank of H1 = 1
since a circle has a single 1-d component
Rank of H2 = 0
since we don’t have any 2-d circles.
This data set consists of 60 points randomly taken from a circle of radius 1.
What should we expect the barcode to look like?
What should we expect the persistence diagram to look like?
Can we use TDA to determine that our points came from a circle

50. Can we use TDA to determine that our points came from a circle
The homology of a circle is as follows:
Rank of H0 = 1 since a circle has only one component
Thus we expect 1 persistent (long) bar in the 0-dim barcode
plus some shorter bars that we can “ignore”
Rank of H1 = 1: circle has a single 1-d cycle that does not bound surface
Thus we expect 1 persistent (long) bar in the 1-dim barcode
plus possible some shorter bars that we can “ignore”
Rank of H2 = 0 since we don’t have any 2-d ccles.
Thus we expect 0 persistent (long) bars in the 2-dim barcode

51. The homology of a circle is as follows:
Rank of H0 = 1 since a circle has only one component
Thus we expect 1 persistent (long) bar in the 0-dim barcode
plus some shorter bars that we can “ignore”
Rank of H1 = 1: circle has a single 1-d cycle that does not bound surface
Thus we expect 1 persistent (long) bar in the 1-dim barcode
plus possible some shorter bars that we can “ignore”
Rank of H2 = 0 since we don’t have any 2-d ccles.
Thus we expect 0 persistent (long) bars in the 2-dim barcode

52. Our data set = 60 points randomly taken from a circle of radius 1
Can you determine from the barcode that our data set came from a circle?
Do you see 1 persistent 0-dim cycle?
Do you see 1 persistent 1-dim cycle?
Do you see 0 persistent 2-dim cycle?
Does the persistent diagram make sense?
1 black point (cycle in H0) is far from the diagonal, while remaining black points are “close” to diagonal
1 red point (cycle in H1) is far from diagonal
All blue points (cycles in H2) are close to diagonal

53. From the barcode:
1 persistent 0-dim cycle  H0 = 1
1 persistent 1-dim cycle  H1 = 1
No persistent 2-dim cycles  H2 = 0
Ignore bars with “small” length
Definition of “small” depends on dimension, data set, application, etc.
From the persistent diagram:
1 black point far from the diagonal  H0 = 1
1 red point far from diagonal  H1 = 1
All blue points close to diagonal  H2 = 0
Ignore points “close” to diagonal
Definition of “close” depends on dimension, data set, application, etc.

54. How does background noise affect persistent diagram?
Next week in B5 MLH you can explore the difference between circle and (circle + various amounts of noise)
How does background noise affect persistent diagram?

55. What does noise look like?

56. What does noise look like?

57. What does noise look like?

58. What does noise look like?

59. What does noise look like?

60. What does noise look like?

61. What does noise look like?

62. What does noise look like?

63. What does noise look like?

64. What does noise look like?

66. Stability

67. How does background noise affect persistent diagram?
Next week in B5 MLH you can explore the difference between circle and (circle + various amounts of noise)
How does background noise affect persistent diagram?

68. f: X  R f-1(-∞, r) (bi, di) Discrete & Computational Geometry
January 2007, Volume 37, Issue 1, pp
Stability of Persistence Diagrams
David Cohen-Steiner, Herbert Edelsbrunner, John Harer
f: X  R
f-1(-∞, r)
(bi, di)

69. Bottleneck Distance.
Let Diag1and Diag2 be persistence diagrams.
The bottleneck distance is
the infimum over all bijections h: Diag1  Diag 2 of
supi d(i; h(i)).

70. || (x1,…,xn) – (y1,…,yn) ||∞ = max{|x1 – y1|,…,|xn - yn|}
Given sets X, Y and bijection b: X  Y,
Bottleneck Distance: dB(X, Y) = inf sup || x – b(x) ||∞
b x

71. || (x1,…,xn) – (y1,…,yn) ||∞ = max{|x1 – y1|,…,|xn - yn|}
Given sets X, Y and bijection g: X  Y,
Bottleneck Distance: dB(X, Y) = inf sup || x – g(x) ||∞
g x

72. dB(D(f), D(g)) ≤ || f − g ||∞ = sup{|f(x) – g(x)|}
Stability theorem:
Let X be a triangulable space with continuous
tame functions f, g : X  R. Then the persistence diagrams, D(f)and D(g), satisfy
dB(D(f), D(g)) ≤ || f − g ||∞ = sup{|f(x) – g(x)|} x

73. Wq(X, Y) = [inf S || x – g(x) ||∞]1/q
|| (x1,…,xn) – (y1,…,yn) ||∞ = max{|x1 – y1|,…,|xn - yn|}
Given sets X, Y and bijection g: X  Y,
Wasserstein distance:
Wq(X, Y) = [inf S || x – g(x) ||∞]1/q q
g x in X

74. Wq(X, Y) = [inf S || x – b(x) ||∞]1/q
|| (x1,…,xn) – (y1,…,yn) ||∞ = max{|x1 – y1|,…,|xn - yn|}
Given sets X, Y and bijection b : X  Y,
Wasserstein distance:
Wq(X, Y) = [inf S || x – b(x) ||∞]1/q q
b x in X

75. (Wasserstein distance)
(Wasserstein distance). The p-th Wasserstein distance between two persistence diagrams,
d1 and d2, is defined as
where g ranges over all bijections from d1 to d2.

76. Wq(D(f), D(g)) ≤ C|| f − g ||∞
Stability theorem:
Let X be a triangulable space whose triangulations grow polynomially with constant exponent j.
Let f, g : X  R be tame Lipschitz functions .
Then there are constants C and k > j no smaller than 1 such that persistence diagrams, D(f) and D(g), satisfy
Wq(D(f), D(g)) ≤ C|| f − g ||∞
for every q ≥ k.
1 – k/q

77. How does background noise affect persistent diagram?
Next week in B5 MLH you can explore the difference between circle and (circle + various amounts of noise)
How does background noise affect persistent diagram?

78. > print( bottleneck(Diag1, Diag2, dimension=0) )
[1]
> print( wasserstein(Diag1, Diag2, p=2, dimension=0) )
[1]
> print( bottleneck(Diag1, Diag2, dimension=1) )
[1]
> print( wasserstein(Diag1, Diag2, p=2, dimension=1) )
[1]

79. http://www. plosone. org/article/info%3Adoi%2F10. 1371%2Fjournal. pone
2008
And section 9.1 in Computational Topology: An Introduction By Herbert Edelsbrunner, John Harer

80. Goal: To determine what genes are involved in a particular periodic pathway
Application: segmentation clock of mouse embryo.
1 somite develops about every 2 hours
What genes are involved in somite development?

81. Persistence:
For each of 7549 genes, create
fk: S1  R, k = 1, …, 7549
fk (time point i) = amount of RNA at time point i for gene k
If gene k is involved in somite development, then fk should have period 2

82. Not period 2:

83. Not period 2:

84. Not period 2:

85. Not period 2:

86. Period 2

87. Persistence:
For each of 7549 genes, create
fk: S1  R, k = 1, …, 7549
fk (time point i) = amount of RNA at time point i for gene k

88. Figure 8. Function g(x) for the expression pattern of Axin2.
Dequéant M-L, Ahnert S, Edelsbrunner H, Fink TMA, et al. (2008) Comparison of Pattern Detection Methods in Microarray Time Series of the Segmentation Clock. PLoS ONE 3(8): e2856. doi: /journal.pone

89. Not period 2:

90. Data from:

91. During the formation of each somite, Lfng is expressed in the PSM as a wave that sweeps across the tissue in a posterior-to-anterior direction (1). Therefore, by visually comparing the anteroposterior position of the Lfng expression stripes in the PSM in stained embryos, it is possible to define an approximate chronological order of the embryos along the segmentation clock oscillation cycle (3, 4). We collected PSM samples from 40 mouse embryos ranging from 19 to 23 somites and used their Lfng expression patterns as a proxy to select 17 samples covering an entire oscillation cycle.
Indeed, due to technical issues, the right PSM samples of the time series were dissected from mouse embryos belonging to five consecutive somite cycles, and they were ordered based on their phase of Lfng expression pattern (revealed by in situ hybridization on the left PSM of each dissected mouse embryo) to reconstitute a unique oscillation cycle [5].

92. Fig. 2. Identification of cyclic genes based on the PSM microarray time series.
Identification of cyclic genes based on the PSM microarray time series. (A) Left side of the 17 mouse embryos, whose right posterior PSMs (below red hatched line) were dissected for microarray analysis. Embryos were ordered along one segmentation clock cycle according to the position of Lfng stripes in their left PSM as revealed by in situ hybridization (fig. S1). (B) Log2 ratios of the expression levels of the Hes1 (blue) and Axin2 (red) cyclic genes in each microarray of the time series. (C) Phaseogram of the cyclic genes identified by microarray and L-S analysis. Blue, decrease in gene expression; yellow, increase in gene expression; pink squares, genes validated by in situ hybridization; and orange circles, nonvalidated genes, that is, not evidently cyclical as detected by in situ hybridization.
M Dequéant et al. Science 2006;314:
Published by AAAS

93.
accession number E-TABM-163

95. Persistence:
For each of 7549 genes, create
fk: S1  R, k = 1, …, 7549
fk (time point i) = amount of RNA at time point i for
gene k

96. g(xi) =[ p(fk)(xi) – 1] / (17 – 1), for i = 1, …, 17.
17 time points  17 equally space time points
microarry expression of gene k at time i  ranked
order of microarry expression of gene k at time i
(0.41, 0.63, 0.11, 0.23, 0.59, …)  (3, 5, 1, 2, 4, …).
fk (time point i) = RNA intensity at time point i for
gene k.
p(fk) = replace RNA intensity with rank order.
g(xi) =[ p(fk)(xi) – 1] / (17 – 1), for i = 1, …, 17.
g(x) obtained by linear interpolation for x ≠ xi for some i Note: 0 ≤ g(x) ≤ 1

97. Figure 8. Function g(x) for the expression pattern of Axin2.
g: S1  R
( 14, )
( 0, )
( 15, )
( 3, )
( 13, )
( 1, )
( 2, )
( 12, )
( 10, )
( 16, )
( 4, )
( 11, )
( 8, )
( 9, )
( 5, )
( 7, )
( 6, 0 )
Dequéant M-L, Ahnert S, Edelsbrunner H, Fink TMA, et al. (2008) Comparison of Pattern Detection Methods in Microarray Time Series of the Segmentation Clock. PLoS ONE 3(8): e2856. doi: /journal.pone

99. Not period 2 implies Φq(f) large.

100. http://journals. plos. org/plosone/article. id=10. 1371/journal. pone

101. L=Lomb-Scargle analysis;
L=Lomb-Scargle analysis;
P=Phase consistencyA=Address reduction; C= Cyclo-hedron test;
S=Stable persistence
The benchmark cyclic genes in bold were identified independ-ently from the microarray analysis

102. Figure 1. Identification of benchmark cyclic genes in the top 300 probe set lists of the five methods.
Dequéant M-L, Ahnert S, Edelsbrunner H, Fink TMA, et al. (2008) Comparison of Pattern Detection Methods in Microarray Time Series of the Segmentation Clock. PLoS ONE 3(8): e2856. doi: /journal.pone

103. Figure 2. Comparison of the intersection of the top 300 ranked probe sets from the five methods.
Dequéant M-L, Ahnert S, Edelsbrunner H, Fink TMA, et al. (2008) Comparison of Pattern Detection Methods in Microarray Time Series of the Segmentation Clock. PLoS ONE 3(8): e2856. doi: /journal.pone

104. Figure 3. Clustering analysis of the top 300 ranked probe sets from the five methods.
Dequéant ML, Ahnert S, Edelsbrunner H, Fink TMA, Glynn EF, et al. (2008) Comparison of Pattern Detection Methods in Microarray Time Series of the Segmentation Clock. PLoS ONE 3(8): e2856. doi: /journal.pone

105. Table 1. Composition of the Wnt Clusters of the Five Methods.
Dequéant ML, Ahnert S, Edelsbrunner H, Fink TMA, Glynn EF, et al. (2008) Comparison of Pattern Detection Methods in Microarray Time Series of the Segmentation Clock. PLoS ONE 3(8): e2856. doi: /journal.pone

106. Persistent homology results are stable:
Add noise to data does not change barcodes significantly.

108. Figure 8. Function g(x) for the expression pattern of Axin2.
Persistent homology results are stable:
Add noise to data does not change barcodes significantly.
Figure 8. Function g(x) for the expression pattern of Axin2.
Dequéant M-L, Ahnert S, Edelsbrunner H, Fink TMA, et al. (2008) Comparison of Pattern Detection Methods in Microarray Time Series of the Segmentation Clock. PLoS ONE 3(8): e2856. doi: /journal.pone

109. || (x1,…,xn) – (y1,…,yn) ||∞ = max{|x1 – y1|,…,|xn - yn|}
Given sets X, Y and bijection g: X  Y,
Bottleneck Distance: dB(X, Y) = inf sup || x – g(x) ||∞
g x

110. where g ranges over all bijections from d1 to d2.
(Wasserstein distance). The p-th Wasserstein distance between two persistence diagrams,
d1 and d2, is defined as
where g ranges over all bijections from d1 to d2.
Probability measures on the space of persistence
diagrams
Yuriy Mileyko1, Sayan Mukherjee2 and John Harer1

111. Persistent homology results are stable:
Add noise to data does not change barcodes significantly.

112. But are the results in this case stable?

113. Perseus:

114. Click on Here to download

115. Downloads idarcy$ cd perseus_3_beta
perseus_3_beta idarcy$ g++ Pers.cpp -O3 -o perseus
Use any text editor (such as TextEdit or pico) to create input File. For example,
perseus_3_beta idarcy$ pico DistanceMatrix
Enter text
If you use pico, use control-X to exit and choose y to save (or control-O to save).

116. Downloads idarcy$ cd perseus_3_beta
perseus_3_beta idarcy$ g++ Pers.cpp -O3 -o perseus
perseus_3_beta idarcy$ pico DistanceMatrix 3
Number of data points.
I.e., size of matrix is 3x3
distance matrix
initial radius r = 0,
step size s = 0.1,
number of steps N = 5,
dimension cap C = 2 
Increase radius by 0.1 five times.
max dim of simplices

117. perseus_3_beta idarcy$ ./perseus rips DistanceMatrix
Read 2 point/radius pairs and birth times!
Writing Cell Complex From RIPS Complex
Done!Complex stored with 2 cells!
+++coreductions: 2 -> 2, fraction removed 0 at height 1
+++reductions: > 2, fraction removed 0 at height 2
Computing Persistence Intervals!
Linearly ordered 2 cells...
Frame [0]: 1
Frame [2]: 2
Done!!! Please consult [output*.txt] for results.