1. Approximating Edit Distance in Near-Linear Time
Alexandr Andoni
(MIT)
Joint work with Krzysztof Onak (MIT)

2. Edit Distance For two strings x,y  ∑n
ed(x,y) = minimum number of edit operations to transform x into y
Edit operations = insertion/deletion/substitution
Important in: computational biology, text processing, etc
Example:
ED( ,
) = 2

3. Computing Edit Distance
Problem: compute ed(x,y) for given x,y{0,1}n
Exactly:
O(n2) [Levenshtein’65]
O(n2/log2 n) for |∑|=O(1) [Masek-Paterson’80]
Approximately in n1+o(1) time:
n1/3+o(1) approximation [Batu-Ergun-Sahinalp’06], improving over [Sahinalp-Vishkin’96, Cole-Hariharan’02, BarYossef-Jayram-Krauthgamer-Kumar’04]
Sublinear time:
≤n1-ε vs ≥n/100 in n1-2ε time [Batu-Ergun-Kilian-Magen-Raskhodnikova-Rubinfeld-Sami’03]

4. Computing via embedding into ℓ1
Embedding: f:{0,1}n → ℓ1
such that ed(x,y) ≈ ||f(x) - f(y)||1
up to some distortion (=approximation)
Can compute ed(x,y) in time to compute f(x)
Best embedding by [Ostrovsky-Rabani’05]:
distortion = 2Õ(√log n)
Computation time: ~n2 randomized (and similar dimension)
Helps for nearest neighbor search, sketching, but not computation…

5. Our result Theorem: Can compute ed(x,y) in
n*2Õ(√log n) time with
2Õ(√log n) approximation
While uses some ideas of [OR’05] embedding, it is not an algorithm for computing the [OR’05] embedding

6. Review of Ostrovsky-Rabani embedding
φm = embedding of strings of length m
δ(m) = distortion of φm
Embedding is recursive
Partition into b blocks (b later chosen to be exp(√log m))
Use embeddings φk for k ≤ m/b
Embed each block separately as follows… X=
m/b

7. Ostrovsky-Rabani embedding (II)X= s
E2s
E3s
Ebs
E1s= rec. embedding of the s substrings
Want to approximate ed(x,y) by
∑i=1..b ∑sS TEMDs(Eis(x), Eis(y))
EMD(A,B) = min-cost bipartite matching
Finish by embedding TEMD into ℓ1 with small distortion T
(thresholded)

8. Distortion of [OR] embedding
Suppose can embed TEMD into ℓ1 with distortion (log m)O(1)
Then [Ostrovsky-Rabani’05] show that distortion of φm is
δ(m) ≤ (log m)O(1) * [δ(m/b) + b]
For b=exp[√log m]
δ(m) ≤ exp[Õ(√log m)]

9. Why it is expensive to compute [OR] embedding
E1s= rec. embedding of the s substrings
In first step, need to compute recursive embedding for ~n/b strings of length ~n/b
The dimension blows up

10. Our Algorithm For each length m in some fixed set L[n],y i z=
z[i:i+m]
For each length m in some fixed set L[n],
compute vectors vimℓ1 such that
||vim – vjm||1 ≈ ed( z[i:i+m], z[j:j+m] )
up to distortion δ(m)
Dimension of vim is only O(log2 n)
Vectors vim are computed inductively from vik for k≤m/b (kL)
Output: ed(x,y)≈||v1n/2 – vn/2+1n/2||1 (i.e., for m=n/2=|x|=|y|)

11. Idea: intuition ||vim – vjm||1 ≈ ed( z[i:i+m], z[j:j+m] )
For each mL, compute φm(z[i:i+m])
as in the O-R recursive step except we use vectors vik, k<m/b & kL, in place of recursive embeddings of shorter substrings (sets Eis)
Resulting φm(z[i:i+m]) have high dimension, >m/b…
Use Bourgain’s Lemma to vectors φm(z[i:i+m]), i=1..n-m,
[Bourgain]: given n vectors qi, construct n vectors q̃i of O(log2 n) dimension such that ||qi-qj||1 ≈ ||q̃i-q̃j||1 up to O(log n) distortion.
Apply to vectors φm(z[i:i+m]) to obtain vectors vim of polylogaritmic dimension
incurs O(log n) distortion at each step of recursion. but OK as there are only ~√log n steps, giving an additional distortion of only exp[Õ(√log n)]

12. Idea: implementation Essential step is:
Main Lemma: fix n vectors viℓ1, of dimension p=O(log2n).
Let s<n. Define Ai={vi, vi+1, …, vi+s-1}.
Then we can compute vectors qiℓ1k for k=O(log2n) such that
||qi – qj||1≈ TEMD(Ai, Aj) up to distortion logO(1) n
Computing qi’s takes Õ(n) time.

13. Proof of Main Lemma Graph-metric: shortest path on a weighted graph
TEMD over n sets Ai
O(log2 n)
Graph-metric: shortest path on a weighted graph
Sparse: Õ(n) edges
“low” = logO(1) n
mink M is semi-metric on Mk with “distance”
dmin,M(x,y)=mini=1..kdM(xi,yi)
minlow ℓ1high
O(1)
minlow ℓ1low
O(log n)
minlow tree-metric
O(log3n)
sparse graph-metric
[Bourgain]
(efficient)
O(log n)
ℓ1low

14. Step 1 TEMD over n sets Ai minlow ℓ1high
O(log2 n)
minlow ℓ1high
Lemma 1: can embed TEMD over n sets in ({0..M}p, ℓ1) into minO(log n) ℓ1M^p with O(log2n) distortion, w.h.p.
Use [A-Indyk-Krauthgamer’08]
(similar to Ostrovsky-Rabani embedding)
Embedding: for each Δ = powers of 2
impose a randomly-shifted grid
one coordinate per cell, equal
to # of points in the cell
Theorem [AIK]:
no contraction w.h.p.
expected expansion = O(log2 n)
Just repeat O(log n) times 

15. minlow ℓ1high
Step 2
O(1)
minlow ℓ1low
Lemma 2: can embed an n point set from ℓ1M into minO(log n) ℓ1k, for k=O(log3 n), with O(1) distortion.
Use (weak) dimensionality reduction in ℓ1
Thm [Indyk’06]: Let A be matrix of size M by k=O(log3 n) with each element chosen from Cauchy distribution. Then for any x̃=Ax, ỹ=Ay:
no contraction: ||x̃-ỹ||1≥||x-y||1 (w.h.p.)
5-expansion: ||x̃-ỹ||1≤5*||x-y||1 (with 0.01 probability)
Just use O(log n) of such embeddings

16. Efficiency of Step 1+2
From step 1+2, we get some embedding f() of sets Ai={vi, vi+1, …, vi+s-1} into minlow ℓ1low
Naively would take Ω(n*s)=Ω(n2) time to compute all f(Ai)
More efficiently:
Note that f() is linear: f(A) = ∑aA f(a)
Then f(Ai) = f(Ai-1)-f(vi-1)+f(vi+s-1)
Compute f(Ai) in order, for a total of Õ(n) time

17. Step 3
minlow ℓ1low
O(log n)
minlow tree-metric
Lemma 3: can embed ℓ1 over {0..M}p into minO(log^2 n) tree-m, with O(log n) distortion.
For each Δ = a power of 2, take O(log n) random grids. Each grid gives a min-coordinate ∞ Δ 

18. minlow tree-metric
Step 4
O(log3n)
sparse graph-metric
Lemma 4: suppose have n points in minlow tree-m, which approximates a metric up to distortion D. Can embed into a graph-metric of size Õ(n) with distortion D.

19. Step 5
sparse graph-metric
O(log n)
ℓ1low
Lemma 5: Given a graph with m edges, can embed the graph-metric into ℓ1low with O(log n) distortion in Õ(m) time.
Just implement Bourgain’s embedding:
Choose O(log2 n) sets Bi
Need to compute the distance from each node to each Bi
For each Bi can compute its distance to each node using Dijkstra’s algorithm in Õ(m) time

20. Summary of Main Lemma
TEMD over n sets Ai
Min-product helps to get low dimension (~small-size sketch)
bypasses impossibility of dim-reduction in ℓ1
Ok that it is not a metric, as long as it is close to a metric
O(log2 n)
minlow ℓ1high
O(1)
oblivious
minlow ℓ1low
O(log n)
minlow tree-metric
O(log3n)
sparse graph-metric
non-oblivious
O(log n)
ℓ1low

21. Conclusion + a question
Theorem: can compute ed(x,y) in
n*2Õ(√log n) time with 2Õ(√log n) approximation
Question: can we do the following “oblivious” dimensionality reduction in ℓ1
Given n, construct a randomized embedding φ:ℓ1M→ℓ1polylog n such that for any v1…vnℓ1M, with high probability, φ has distortion logO(1) n on these vectors?
If φ exists, it cannot be linear [Charikar-Sahai’02]