Download presentation

Presentation is loading. Please wait.

1
**BiG-Align: Fast Bipartite Graph Alignment**

Danai Koutra Hanghang Tong David Lubensky IEEE ICDM, 7-10 December 2013, Dallas, Texas, USA

2
**Can we identify users across social networks?**

Same or “similar” users? Task: link prediction (friend link), recommendations Danai Koutra (CMU)

3
**protein-protein alignment**

More applications? chemical compound comparison protein-protein alignment Optical character recognition link prediction & viral marketing IR: Information Retrieval link prediction and viral marketing in social networks 2) protein-protein interaction networks 3) optical character recognition 4) chemical compounds comparison 5) structure matching in databases 6) synonyms extraction in information retrieval 7) Wiki translation (graphs of documents from different language – learn automatic translation) IR: synonym extraction wiki translation Structure matching in DB Danai Koutra (CMU)

4
**RoadMap Problem Definition What’s different? BiG-Align Uni-Align**

Conclusions Danai Koutra (CMU)

5
**Problem Definition INPUT: A, B A B u users s e r groups 1 1 0 0 0**

users A B Combinatorial problem Relax the problem + approximate the solution Danai Koutra (CMU)

6
**Problem Definition INPUT: A, B OUTPUT: P and … A B P (users) B A**

(permutation matrices) u s e r groups users A B P (users) A B Combinatorial problem Relax the problem + approximate the solution Output: mappings at two granularities; users and groups/communities Danai Koutra (CMU)

7
**Problem Definition INPUT: A, B OUTPUT: P and Q**

(permutation matrices) s.t. min || PAQ - B|| F 2 u s e r groups users A B constraints / relaxations Graph isomorphism: HARD (P or NP complete?) Subgraph isomorphism: NP-complete And now what? P (users) A B Q (groups) Output: mappings at two granularities; users and groups/communities users/groups permutation of A permutation of users/groups in A Danai Koutra (CMU)

8
**Problem Definition: constraints**

u g … … … … … A B INPUT: A, B OUTPUT: P, Q correspondence matrices s.t. min || PAQ - B|| F 2 P (users) A B Q (groups) Danai Koutra (CMU)

9
**Problem Definition: constraints**

u g … … … … … A B INPUT: A, B OUTPUT: P, Q correspondence matrices s.t. min || PAQ - B|| F 2 CONSTRAINTS: Pij, Qij = probabilities (not 1-1 mapping) sparse matrices P and Q (more efficient for large scale graphs) P (users) A B Q (groups) Danai Koutra (CMU)

10
**RoadMap Problem Definition What’s different? BiG-Align Uni-Align**

Conclusions Danai Koutra (CMU)

11
**What’s different? Focus on bipartite graphs BiG-Align other approaches**

vs. Focus on bipartite graphs Danai Koutra (CMU)

12
**What’s different? Focus on bipartite graphs**

BiG-Align other approaches vs. Focus on bipartite graphs New optimization problem/constraints Danai Koutra (CMU)

13
**What’s different? Focus on bipartite graphs**

BiG-Align other approaches vs. Focus on bipartite graphs New optimization problem/constraints The hope is: the specific graph structure will lead to more accurate graph alignment Danai Koutra (CMU)

14
Why bipartite graphs? ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs Danai Koutra (CMU)

15
**Why bipartite graphs? nodes communities**

ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level nodes communities Danai Koutra (CMU)

16
**Why bipartite graphs? nodes communities**

ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level conversion of uni-partite graph to bi-partite --> clustering + (2) nodes communities Danai Koutra (CMU)

17
**Why bipartite graphs? nodes communities users time email**

ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level conversion of unipartite graph to bipartite --> clustering + (2) general formulation: match clouds of points (point-feature graph) tensors (e.g. time-evolving, or other 3rd dimension) nodes communities users time Danai Koutra (CMU)

18
**RoadMap Problem Definition What’s different? BiG-Align Uni-Align**

Conclusions Danai Koutra (CMU)

19
**alternating, projected**

BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent until convergence Danai Koutra (CMU)

20
**BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q)**

Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Probabilistic Constraint until convergence Danai Koutra (CMU)

21
**BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q)**

Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Sparsity Constraint until convergence Danai Koutra (CMU)

22
**min f = min||| PAQ – B||F 2 + λΣPij + μΣQij**

BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Sparsity Constraint until convergence min f = min||| PAQ – B||F λΣPij + μΣQij Danai Koutra (CMU)

23
**RoadMap Problem Definition What’s different? BiG-Align Uni-Align**

Optimizations Uni-Align Conclusions Danai Koutra (CMU)

24
**BiG-Align: Optimizations**

Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent until convergence Danai Koutra (CMU)

25
**BiG-Align: Optimizations**

Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

26
**Optimization 1: Structurally equivalent nodes**

Details Aggregation to super-nodes Graph A Danai Koutra (CMU)

27
**BiG-Align: Optimizations**

Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

28
**Optimization 2: Initialization of P and Q**

Details Why is the initialization important? global minimum local minima … Danai Koutra (CMU)

29
**Optimization 2: Initialization of P and Q**

Details Social networks are structured: the degree distribution is power-law like. ranked nodes log(degree) Social networks are structured: the degree distribution of the nodes is power-law like. Danai Koutra (CMU)

30
**Optimization 2: Initialization of P and Q**

Details 1-1 matching of clusters of degrees Network-inspired initialization degree rank of node knee k 1-1 matching of top k nodes user degrees of GA user degrees in GB … . 2000 1500 1000 945 940 800 799 750 740 735 730 3 2 1 500 450 449 445 P k cluster 1 cluster 2 cluster n Top k nodes with significantly higher degree -> handle them separately from the low degree nodes cluster 1 cluster 2 cluster n Danai Koutra (CMU)

31
**BiG-Align: Optimizations**

Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

32
**Optimization 3: Steps of gradient descent**

Details Constant step: thrashing or slow convergence Danai Koutra (CMU)

33
**Optimization 3: Steps of gradient descent**

Details Variable step with line search: strategy for local optimum ηP = argmin f(ηP) = g1(P,Q,A,B) ηQ = argmin f(ηQ) = g2(P,Q,A,B) closed formulas Danai Koutra (CMU)

34
**Optimization 3: Steps of gradient descent**

Details Variable step with line search: strategy for local optimum BiG-Align-Exact: computes the steps at every iteration ηP = argmin f(ηP) = g1(P,Q,A,B) ηQ = argmin f(ηQ) = g2(P,Q,A,B) closed formulas Danai Koutra (CMU)

35
**Optimization 3: Steps of gradient descent**

Details But 3.104 104 2.104 10-4 step size (η) iterations Slow change in the steps Danai Koutra (CMU)

36
**Optimization 3: Steps of gradient descent**

Details But BiG-Align-Skip: compute η’s every m (=500) iterations 3.104 104 2.104 10-4 step size (η) iterations Slow change in the steps Danai Koutra (CMU)

37
**RoadMap Problem Definition What’s different? BiG-Align Uni-Align**

Experiments Uni-Align Conclusions Danai Koutra (CMU)

38
**Experimental Setup Implementation: Matlab**

Dataset: IMDB movie-genre graph and subgraphs (1027 movies x 27 genres) Setup: random permutations noise level: % Ground truth Simulate real-world applications Danai Koutra (CMU)

39
**State-of-the-art Background Umeyama’s algorithm [Umeyama88]:**

SVD-based NMF-based approach [Ding+08]: Builds on top of Umeyama’s approach Net-Align [Bayati+09] Belief Propagation Danai Koutra (CMU)

40
**State-of-the-art Background Umeyama’s algorithm [Umeyama88]:**

SVD-based NMF-based approach [Ding+08]: Builds on top of Umeyama’s approach Net-Align [Bayati+09] Belief Propagation Bi-partite Uni-partite Danai Koutra (CMU)

41
**Big-Align: Accuracy vs. Runtime**

Umeyama NetAlign NMF-based BiG-Align exact BiG-Align skip Small marker size: 300 nodes Big marker size: 700 nodes marker size related to graph size Danai Koutra (CMU)

42
**Big-Align: Accuracy vs. Runtime**

Umeyama NetAlign NMF-based BiG-Align exact BiG-Align skip Big-Align improves both speed and accuracy. Danai Koutra (CMU)

43
**Big-Align: Accuracy w.r.t. noise**

BiG-Align-exact BiG-Align-skip NMF-based NetAlign-deg NetAlign-full Umeyama Danai Koutra (CMU)

44
**Big-Align: Accuracy w.r.t. noise**

BiG-Align-exact BiG-Align-skip NMF-based NetAlign-deg NetAlign-full Umeyama BiG-Align improves the accuracy for almost all levels of noise. Danai Koutra (CMU)

45
**RoadMap Problem Definition What’s different? BiG-Align Uni-Align**

Conclusions Danai Koutra (CMU)

46
**Algorithm: Uni-Align Details n nodes d features min || PAQ - B||F 2 P**

fixed node degree clustering coeff … … Danai Koutra (CMU)

47
**Algorithm: Uni-Align Details n nodes d features O(n.d2)**

min || PAQ - B||F 2 P P = g*(A,B,S,U)= = closed-form solution SVD A = USVT O(n.d2) Explain it somehow Danai Koutra (CMU)

48
**RoadMap Problem Definition What’s different? BiG-Align Uni-Align**

Experiments Conclusions Danai Koutra (CMU)

49
**Uni-Align Dataset: Facebook friendship graph (64K users)**

Setup: uni-partite bi-partite graph Feature extraction node degree egonet degree edges in egonet mean degree of node’s neighbors egonet Danai Koutra (CMU)

50
**Uni-Align: Accuracy vs. Runtime**

NMF-based NetAlign Umeyama Uni-Align, followed by Net-Align, is more accurate and faster than other approaches. Danai Koutra (CMU)

51
**Uni-Align: Runtime Uni-Align**

Umeyama Uni-Align NMF-based NetAlign-deg NetAlign-full Uni-Align is 2x - 31,700x faster depending on graph size. Danai Koutra (CMU)

52
**RoadMap Problem Definition What’s different? BiG-Align Uni-Align**

Conclusions Danai Koutra (CMU)

53
Conclusions Formulation: new problem / constraints Danai Koutra (CMU)

54
**Conclusions Formulation: new problem / constraints Algorithms:**

BiG-Align: optimized alternating projected gradient descent Uni-Align: alignment for uni-partite graphs Danai Koutra (CMU)

55
**Conclusions Formulation: new problem / constraints Algorithms:**

BiG-Align: optimized alternating projected gradient descent Uni-Align: alignment for uni-partite graphs Evaluations: more accurate and efficient Danai Koutra (CMU)

56
**Beyond BiG-Align: Multi-way Linkage**

~ S1: Dynamic Graph Linkage S2: Community-level Linkage ~ ~ beyond this paper, we found that big-align forms a core building block for several other more complicated matching task, such as we will not go into the details, but the bottom line is that all of them can be built upon big-align. and this also led to 7 patents S3: Hetero. Graph Linkage S4: Multi-relational DB Linkage All build upon BiG-Align (2) Led to 7 patents Danai Koutra (CMU)

57
**Thank you! http://www.cs.cmu.edu/~dkoutra danai@cs.cmu.edu**

Danai Koutra (CMU)

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google