# BiG-Align: Fast Bipartite Graph Alignment

## Presentation on theme: "BiG-Align: Fast Bipartite Graph Alignment"— Presentation transcript:

BiG-Align: Fast Bipartite Graph Alignment
Danai Koutra Hanghang Tong David Lubensky IEEE ICDM, 7-10 December 2013, Dallas, Texas, USA

Can we identify users across social networks?

protein-protein alignment
More applications? chemical compound comparison protein-protein alignment Optical character recognition link prediction & viral marketing IR: Information Retrieval link prediction and viral marketing in social networks 2) protein-protein interaction networks 3) optical character recognition 4) chemical compounds comparison 5) structure matching in databases 6) synonyms extraction in information retrieval 7) Wiki translation (graphs of documents from different language – learn automatic translation) IR: synonym extraction wiki translation Structure matching in DB Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

Problem Definition INPUT: A, B A B u users s e r groups 1 1 0 0 0
users A B Combinatorial problem Relax the problem + approximate the solution Danai Koutra (CMU)

Problem Definition INPUT: A, B OUTPUT: P and … A B P (users) B A
(permutation matrices) u s e r groups users A B P (users) A B Combinatorial problem Relax the problem + approximate the solution Output: mappings at two granularities; users and groups/communities Danai Koutra (CMU)

Problem Definition INPUT: A, B OUTPUT: P and Q
(permutation matrices) s.t. min || PAQ - B|| F 2 u s e r groups users A B constraints / relaxations Graph isomorphism: HARD (P or NP complete?) Subgraph isomorphism: NP-complete And now what? P (users) A B Q (groups) Output: mappings at two granularities; users and groups/communities users/groups permutation of A permutation of users/groups in A Danai Koutra (CMU)

Problem Definition: constraints
u g … … … … … A B INPUT: A, B OUTPUT: P, Q correspondence matrices s.t. min || PAQ - B|| F 2 P (users) A B Q (groups) Danai Koutra (CMU)

Problem Definition: constraints
u g … … … … … A B INPUT: A, B OUTPUT: P, Q correspondence matrices s.t. min || PAQ - B|| F 2 CONSTRAINTS: Pij, Qij = probabilities (not 1-1 mapping) sparse matrices P and Q (more efficient for large scale graphs) P (users) A B Q (groups) Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

What’s different? Focus on bipartite graphs BiG-Align other approaches
vs. Focus on bipartite graphs Danai Koutra (CMU)

What’s different? Focus on bipartite graphs
BiG-Align other approaches vs. Focus on bipartite graphs New optimization problem/constraints Danai Koutra (CMU)

What’s different? Focus on bipartite graphs
BiG-Align other approaches vs. Focus on bipartite graphs New optimization problem/constraints The hope is: the specific graph structure will lead to more accurate graph alignment Danai Koutra (CMU)

Why bipartite graphs? ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs Danai Koutra (CMU)

Why bipartite graphs? nodes communities
ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level nodes communities Danai Koutra (CMU)

Why bipartite graphs? nodes communities
ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level conversion of uni-partite graph to bi-partite --> clustering + (2) nodes communities Danai Koutra (CMU)

Why bipartite graphs? nodes communities users time email
ubiquitous – e.g., users-files, authors-papers, customers-products, users-msg/groups user-movie rating graphs coupled alignment: individual & community-level conversion of unipartite graph to bipartite --> clustering + (2) general formulation: match clouds of points (point-feature graph) tensors (e.g. time-evolving, or other 3rd dimension) nodes communities users time Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

alternating, projected
BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent until convergence Danai Koutra (CMU)

BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q)
Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Probabilistic Constraint until convergence Danai Koutra (CMU)

BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q)
Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Sparsity Constraint until convergence Danai Koutra (CMU)

min f = min||| PAQ – B||F 2 + λΣPij + μΣQij
BiG-Align: algorithm DeTaIlS INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q Sparsity Constraint until convergence min f = min||| PAQ – B||F λΣPij + μΣQij Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Optimizations Uni-Align Conclusions Danai Koutra (CMU)

BiG-Align: Optimizations
Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent until convergence Danai Koutra (CMU)

BiG-Align: Optimizations
Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

Optimization 1: Structurally equivalent nodes
Details Aggregation to super-nodes Graph A Danai Koutra (CMU)

BiG-Align: Optimizations
Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

Optimization 2: Initialization of P and Q
Details Why is the initialization important? global minimum local minima Danai Koutra (CMU)

Optimization 2: Initialization of P and Q
Details Social networks are structured: the degree distribution is power-law like. ranked nodes log(degree) Social networks are structured: the degree distribution of the nodes is power-law like. Danai Koutra (CMU)

Optimization 2: Initialization of P and Q
Details 1-1 matching of clusters of degrees Network-inspired initialization degree rank of node knee k 1-1 matching of top k nodes user degrees of GA user degrees in GB . 2000 1500 1000 945 940 800 799 750 740 735 730 3 2 1 500 450 449 445 P k cluster 1 cluster 2 cluster n Top k nodes with significantly higher degree -> handle them separately from the low degree nodes cluster 1 cluster 2 cluster n Danai Koutra (CMU)

BiG-Align: Optimizations
Details INPUT: A, B initialize(P) initialize(Q) Pk+1 = Pk – ηP df(Pk, Qk)/dP valid_projection(Pk+1) Qk+1 = Qk – ηQ df(Pk+1, Qk)/dQ valid_projection(Qk+1) update(ηP, ηQ) OUTPUT: P, Q alternating, projected gradient descent alternating, projected gradient descent until convergence Danai Koutra (CMU)

Optimization 3: Steps of gradient descent
Details Constant step: thrashing or slow convergence Danai Koutra (CMU)

Optimization 3: Steps of gradient descent
Details Variable step with line search: strategy for local optimum ηP = argmin f(ηP) = g1(P,Q,A,B) ηQ = argmin f(ηQ) = g2(P,Q,A,B) closed formulas Danai Koutra (CMU)

Optimization 3: Steps of gradient descent
Details Variable step with line search: strategy for local optimum BiG-Align-Exact: computes the steps at every iteration ηP = argmin f(ηP) = g1(P,Q,A,B) ηQ = argmin f(ηQ) = g2(P,Q,A,B) closed formulas Danai Koutra (CMU)

Optimization 3: Steps of gradient descent
Details But 3.104 104 2.104 10-4 step size (η) iterations Slow change in the steps Danai Koutra (CMU)

Optimization 3: Steps of gradient descent
Details But BiG-Align-Skip: compute η’s every m (=500) iterations 3.104 104 2.104 10-4 step size (η) iterations Slow change in the steps Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Experiments Uni-Align Conclusions Danai Koutra (CMU)

Experimental Setup Implementation: Matlab
Dataset: IMDB movie-genre graph and subgraphs (1027 movies x 27 genres) Setup: random permutations noise level: % Ground truth Simulate real-world applications Danai Koutra (CMU)

State-of-the-art Background Umeyama’s algorithm [Umeyama88]:
SVD-based NMF-based approach [Ding+08]: Builds on top of Umeyama’s approach Net-Align [Bayati+09] Belief Propagation Danai Koutra (CMU)

State-of-the-art Background Umeyama’s algorithm [Umeyama88]:
SVD-based NMF-based approach [Ding+08]: Builds on top of Umeyama’s approach Net-Align [Bayati+09] Belief Propagation Bi-partite  Uni-partite Danai Koutra (CMU)

Big-Align: Accuracy vs. Runtime
Umeyama NetAlign NMF-based BiG-Align exact BiG-Align skip Small marker size: 300 nodes Big marker size: 700 nodes marker size related to graph size Danai Koutra (CMU)

Big-Align: Accuracy vs. Runtime
Umeyama NetAlign NMF-based BiG-Align exact BiG-Align skip Big-Align improves both speed and accuracy. Danai Koutra (CMU)

Big-Align: Accuracy w.r.t. noise
BiG-Align-exact BiG-Align-skip NMF-based NetAlign-deg NetAlign-full Umeyama Danai Koutra (CMU)

Big-Align: Accuracy w.r.t. noise
BiG-Align-exact BiG-Align-skip NMF-based NetAlign-deg NetAlign-full Umeyama BiG-Align improves the accuracy for almost all levels of noise. Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

Algorithm: Uni-Align Details n nodes d features min || PAQ - B||F 2 P
fixed node degree clustering coeff … … Danai Koutra (CMU)

Algorithm: Uni-Align Details n nodes d features O(n.d2)
min || PAQ - B||F 2 P P = g*(A,B,S,U)= = closed-form solution SVD A = USVT O(n.d2) Explain it somehow Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Experiments Conclusions Danai Koutra (CMU)

Uni-Align Dataset: Facebook friendship graph (64K users)
Setup: uni-partite  bi-partite graph Feature extraction node degree egonet degree edges in egonet mean degree of node’s neighbors egonet Danai Koutra (CMU)

Uni-Align: Accuracy vs. Runtime
NMF-based NetAlign Umeyama Uni-Align, followed by Net-Align, is more accurate and faster than other approaches. Danai Koutra (CMU)

Uni-Align: Runtime Uni-Align
Umeyama Uni-Align NMF-based NetAlign-deg NetAlign-full Uni-Align is 2x - 31,700x faster depending on graph size. Danai Koutra (CMU)

RoadMap Problem Definition What’s different? BiG-Align Uni-Align
Conclusions Danai Koutra (CMU)

Conclusions Formulation: new problem / constraints Danai Koutra (CMU)

Conclusions Formulation: new problem / constraints Algorithms:
BiG-Align: optimized alternating projected gradient descent Uni-Align: alignment for uni-partite graphs Danai Koutra (CMU)

Conclusions Formulation: new problem / constraints Algorithms:
BiG-Align: optimized alternating projected gradient descent Uni-Align: alignment for uni-partite graphs Evaluations: more accurate and efficient Danai Koutra (CMU)