Fast Random Walk with Restart and Its Applications Hanghang Tong, Christos Faloutsos and Jia-Yu (Tim) Pan ICDM 2006 Dec. 18-22, HongKong ICDM2006 Dec, 18-22, HongKong

Motivating Questions Q: How to measure the relevance? A: Random walk with restart Q: How to do it efficiently? A: This talk tries to answer! ICDM2006 Dec, 18-22, HongKong

Random walk with restart 1 4 3 2 5 6 7 9 10 8 11 12 ICDM2006 Dec, 18-22, HongKong

Random walk with restart 1 4 3 2 5 6 7 9 10 8 11 12 0.13 0.10 0.05 0.08 0.04 0.02 0.03 Node 4 Node 1 Node 2 Node 3 Node 5 Node 6 Node 7 Node 8 Node 9 Node 10 Node 11 Node 12 0.13 0.10 0.22 0.05 0.08 0.04 0.03 0.02 Nearby nodes, higher scores Ranking vector More red, more relevant ICDM2006 Dec, 18-22, HongKong

Automatic Image Caption Q { } Cat Forest Grass Tiger … { Sea Sun Sky Wave } ? {?, ?, ?,} A: RWR! [Pan KDD2004] ICDM2006 Dec, 18-22, HongKong

Region Image Keyword Test Image Sea Sun Sky Wave Cat Forest Tiger Grass Test Image Keyword ICDM2006 Dec, 18-22, HongKong

Region Image Keyword Test Image {Grass, Forest, Cat, Tiger} Sea Sun Sky Wave Cat Forest Tiger Grass Keyword

Neighborhood Formulation … … Q: what is most related conference to ICDM A: RWR! [Sun ICDM2005] … … Conference Author

NF: example

Center-Piece Subgraph(CePS) Q ? Original Graph Black: query nodes CePS A: RWR! [Tong KDD 2006] ICDM2006 Dec, 18-22, HongKong

CePS: Example ICDM2006 Dec, 18-22, HongKong

Other Applications Content-based Image Retrieval [He] Personalized PageRank [Jeh], [Widom], [Haveliwala] Anomaly Detection (for node; link) [Sun] Link Prediction [Getoor], [Jensen] Semi-supervised Learning [Zhu], [Zhou] … ICDM2006 Dec, 18-22, HongKong

Roadmap Background Basic Idea FastRWR Experimental Results Conclusion RWR: Definitions RWR: Algorithms Basic Idea FastRWR Pre-Compute Stage On-Line Stage Experimental Results Conclusion ICDM2006 Dec, 18-22, HongKong

Computing RWR n x 1 n x n n x 1 1 Restart p Starting vector Ranking vector Adjacent matrix 1 4 3 2 5 6 7 9 10 8 11 12 1 n x 1 n x n n x 1 ICDM2006 Dec, 18-22, HongKong

Fast RWR Finds the Root Solution ! Beyond RWR : Maxwell Equation for Web! [Chakrabarti] P-PageRank [Haveliwala] SM Learning [Zhou, Zhu] RL in CBIR [He] RWR [Pan, Sun] PageRank [Haveliwala] Fast RWR Finds the Root Solution ! ICDM2006 Dec, 18-22, HongKong

Q: Given query i, how to solve it?

OntheFly: Slow on-line response O(mE) 1 4 3 2 5 6 7 9 10 8 11 12 0.13 0.10 0.05 0.08 0.04 0.02 0.03 1 4 3 2 5 6 7 9 10 8 11 12 No pre-computation/ light storage Slow on-line response O(mE) ICDM2006 Dec, 18-22, HongKong

PreCompute R: [Haveliwala] 1 4 3 2 5 6 7 9 10 8 11 10 9 12 2 1 8 3 11 0.13 0.10 0.05 0.08 0.04 0.02 0.03 10 9 12 2 1 8 R: 3 11 4 6 5 7 [Haveliwala] ICDM2006 Dec, 18-22, HongKong

PreCompute: Fast on-line response Heavy pre-computation/storage cost 1 4 3 2 5 6 7 9 10 8 11 12 0.13 0.10 0.05 0.08 0.04 0.02 0.03 1 4 3 2 5 6 7 9 10 8 11 12 Fast on-line response Heavy pre-computation/storage cost O(n ) 3 O(n ) 2 ICDM2006 Dec, 18-22, HongKong

Q: How to Balance? On-line Off-line ICDM2006 Dec, 18-22, HongKong

Roadmap Background Basic Idea FastRWR Experimental Results Conclusion RWR: Definitions RWR: Algorithms Basic Idea FastRWR Pre-Compute Stage On-Line Stage Experimental Results Conclusion ICDM2006 Dec, 18-22, HongKong

Basic Idea Find Community Combine Fix the remaining 1 4 3 2 5 6 7 9 10 8 11 12 1 4 3 2 5 6 7 9 10 8 11 12 Find Community 5 6 7 9 10 8 11 12 5 6 7 9 10 8 11 12 1 4 3 2 5 6 7 9 10 8 11 12 0.13 0.10 0.05 0.08 0.04 0.02 0.03 1 4 3 2 1 4 3 2 1 4 3 2 5 6 7 9 10 8 11 12 1 4 3 2 5 6 7 9 10 8 11 12 Combine Fix the remaining

Pre-computational stage -1 Q: A: A few small, instead of ONE BIG, matrices inversions Efficiently compute and store Q ICDM2006 Dec, 18-22, HongKong

On-Line Query Stage + Q: Efficiently recover one column of Q -1 Q: Efficiently recover one column of Q A: A few, instead of MANY, matrix-vector multiplication + ICDM2006 Dec, 18-22, HongKong

Roadmap Background Basic Idea FastRWR Experimental Results Conclusion RWR: Definitions RWR: Algorithms Basic Idea FastRWR Pre-Compute Stage On-Line Stage Experimental Results Conclusion ICDM2006 Dec, 18-22, HongKong

Pre-compute Stage p1: B_Lin Decomposition p2: Q matrices P1.1 partition P1.2 low-rank approximation p2: Q matrices P2.1 computing (for each partition) P2.2 computing (for concept space) ICDM2006 Dec, 18-22, HongKong

P1.1: partition Within-partition links cross-partition links 1 4 3 2 5 6 7 9 10 8 11 12 10 9 12 2 8 1 3 11 4 6 5 7 Within-partition links cross-partition links ICDM2006 Dec, 18-22, HongKong

P1.1: block-diagonal 1 4 3 2 5 6 7 9 10 8 11 12 10 9 12 2 8 1 3 11 4 6 5 7 ICDM2006 Dec, 18-22, HongKong

P1.2: LRA for ~ |S| << |W2| 1 4 3 2 5 6 7 9 10 8 11 12 10 9 12 2 ICDM2006 Dec, 18-22, HongKong

= +

p2.1 Computing ICDM2006 Dec, 18-22, HongKong

Comparing and = Computing Time Storage Cost 100,000 nodes; 100 partitions Computing 100,00x is Faster! Storage Cost 100x saving! Q 1,1 1,2 1,k =

~ Q: How to fix the green portions? ~ + ~ + ?

p2.2 Computing: -1 Q 1,1 1,2 1,k _ U = V 1 4 3 2 5 6 7 9 10 8 11 12 ICDM2006 Dec, 18-22, HongKong

We have: SM Lemma says: Communities Bridges ICDM2006 Dec, 18-22, HongKong

Roadmap Background Basic Idea FastRWR Experimental Results Conclusion RWR: Definitions RWR: Algorithms Basic Idea FastRWR Pre-Compute Stage On-Line Stage Experimental Results Conclusion ICDM2006 Dec, 18-22, HongKong

? On-Line Stage Q + A (SM lemma) Query Result Pre-Computation ICDM2006 Dec, 18-22, HongKong

On-Line Query Stage q1: q2: q3: q4: q5: q6: ICDM2006 Dec, 18-22, HongKong

ICDM2006 Dec, 18-22, HongKong

Roadmap Background Basic Idea FastRWR Experimental Results Conclusion RWR: Definitions RWR: Algorithms Basic Idea FastRWR Pre-Compute Stage On-Line Stage Experimental Results Conclusion ICDM2006 Dec, 18-22, HongKong

Experimental Setup Dataset Approx. Quality: Relative Accuracy DBLP/authorship Author-Paper 315k nodes 1,800k edges Approx. Quality: Relative Accuracy Application: Center-Piece Subgraph ICDM2006 Dec, 18-22, HongKong

Query Time vs. Pre-Compute Time Log Query Time Quality: 90%+ On-line: Up to 150x speedup Pre-computation: Two orders saving Log Pre-compute Time ICDM2006 Dec, 18-22, HongKong

Query Time vs. Pre-Storage Log Query Time Quality: 90%+ On-line: Up to 150x speedup Pre-storage: Three orders saving Log Storage ICDM2006 Dec, 18-22, HongKong

Roadmap Background Basic Idea FastRWR Experimental Results Conclusion RWR: Definitions RWR: Algorithms Basic Idea FastRWR Pre-Compute Stage On-Line Stage Experimental Results Conclusion ICDM2006 Dec, 18-22, HongKong

Conclusion FastRWR More in the paper Reasonable quality preservation (90%+) 150x speed-up: query time Orders of magnitude saving: pre-compute & storage More in the paper The variant of FastRWR and theoretic justification Implementation details normalization, low-rank approximation, sparse More experiments Other datasets, other applications ICDM2006 Dec, 18-22, HongKong

Q&A Thank you! htong@cs.cmu.edu www.cs.cmu.edu/~htong ICDM2006 Dec, 18-22, HongKong