# Parallel Double Greedy Submodular Maxmization Xinghao Pan, Stefanie Jegelka, Joseph Gonzalez, Joseph Bradley, Michael I. Jordan.

## Presentation on theme: "Parallel Double Greedy Submodular Maxmization Xinghao Pan, Stefanie Jegelka, Joseph Gonzalez, Joseph Bradley, Michael I. Jordan."— Presentation transcript:

Parallel Double Greedy Submodular Maxmization Xinghao Pan, Stefanie Jegelka, Joseph Gonzalez, Joseph Bradley, Michael I. Jordan

Submodular Set Functions F: 2 V → R, such that for all and Diminishing Returns Property

Submodular Examples clustering graph partitioning Max / Min Cut S Graph Algorithms (Krause & Guestrin 2005) Sensing (Kempe, Kleinberg, Tardos 2003, Mossel & Roch 2007) Network Analysis Document Summarization (Lin & Bilmes 2011)

? Concurrency Control Double Greedy Optimal ½ - approximation Bounded overhead Coordination Free Double Greedy Bounded error Minimal overhead Submodular Maximization max F(A), A ⊆ V Monotone (increasing) functions [ Positive marginal gains ] Non-monotone functions Greedy (Nemhauser et al, 1978) (1-1/e) - approximation Optimal polytime G REE D I (Mirzasoleiman et al, 2013) (1-1/e) 2 / p – approximation 1 MapReduce round (Kumar et al, 2013) 1 / (2+ε) – approximation O(1/ε) MapReduce rounds Double Greedy (Buchbinder et al, 2012) ½ - approximation Optimal polytime Sequential Parallel / Distributed

Double Greedy Algorithm Set A Set B u v w x y z u v w xyz

Double Greedy Algorithm Set A Set B u v w x y z u v w xyz

Double Greedy Algorithm Set A Set B u v w x y z + A- B 01 p( | A, B ) = u rand u v w xyz Marginal gains Δ + (u|A) = F(A ∪ u) – F(A), Δ - (u|B) = F(B \ u) – F(B).

Double Greedy Algorithm Set A Set B u v w x y z + A- B 01 p( | A, B ) = u rand u v w xyz Marginal gains Δ + (u|A) = F(A ∪ u) – F(A), Δ - (u|B) = F(B \ u) – F(B).

Double Greedy Algorithm Set A Set B u w x y z + A- B 01 p( | A, B ) = v rand u v w xyz Marginal gains Δ + (v|A) = F(A ∪ v) – F(A), Δ - (v|B) = F(B \ v) – F(B). v v

Double Greedy Algorithm Set A Set B u w x y z u w xyz Return A

Parallel Double Greedy Algorithm Set A Set B u v w x y z u v w xyz CPU 1 CPU 2

Parallel Double Greedy Algorithm Set A Set B u v w x y z u v w x y z CPU 1 CPU 2 uu Δ + (u | ?) = ? Δ - (u | ?) = ? Δ + (v | ?) = ? Δ - (v | ?) = ? vv ? ?

Concurrency Control Double Greedy Set A Set B u v w x y z u v w x y z CPU 1 CPU 2 uu v v Maintain bounds on A, B  Enable threads to make decisions locally

Concurrency Control Double Greedy Set A Set B w x y z u v w x y z CPU 1 CPU 2 uu Δ + (u|A) ∈ [Δ + min (u|A), Δ + max (u|A)] Δ - (u|B) ∈ [Δ - min (u|B), Δ - max (u|B) ] vv Maintain bounds on A, B  Enable threads to make decisions locally Δ + (v|A) ∈ [Δ + min (v|A), Δ + max (v|A)] Δ - (v|B) ∈ [Δ - min (v|B), Δ - max (v|B) ] + A- B 01 p( | A, B ) = u Uncertaint y + A- B 01 p( | A, B ) = v Uncertaint y

Concurrency Control Double Greedy Set A Set B w x y z u v w x y z CPU 1 CPU 2 uu Δ + (u|A) ∈ [Δ + min (u|A), Δ + max (u|A)] Δ - (u|B) ∈ [Δ - min (u|B), Δ - max (u|B) ] vv Maintain bounds on A, B  Enable threads to make decisions locally Δ + (v|A) ∈ [Δ + min (v|A), Δ + max (v|A)] Δ - (v|B) ∈ [Δ - min (v|B), Δ - max (v|B) ] + A- B 01 p( | A, B ) = u rand Uncertaint y + A- B 01 p( | A, B ) = v Uncertaint y u

Concurrency Control Double Greedy Set A Set B w x y z v w x y z CPU 1 CPU 2 uu Δ + (u|A) ∈ [Δ + min (u|A), Δ + max (u|A)] Δ - (u|B) ∈ [Δ - min (u|B), Δ - max (u|B) ] vv Maintain bounds on A, B  Enable threads to make decisions locally Δ + (v|A) ∈ [Δ + min (v|A), Δ + max (v|A)] Δ - (v|B) ∈ [Δ - min (v|B), Δ - max (v|B) ] + A- B 01 p( | A, B ) = u Uncertaint y + A- B 01 p( | A, B ) = v Uncertaint y rand Δ + (v|A) = F(A ∪ v) – F(A) Δ - (v|B) = F(B \ v) – F(B) v rand Common, Fast Rare, Slow

Properties of CC Double Greedy Theorem: CC double greedy is serializable. Corollary: CC double greedy preserves optimal approximation guarantee of ½OPT. Lemma: CC has bounded overhead. Expected number of blocked elements set cover with costs: < 2τ sparse max cut:< 2τ |E| / |V| Correctness Concurren cy Set ASet B w D E z x y x y w uu τ

Empirical Validation Multicore up to 16 threads Set cover, Max graph cut Real and synthetic graphs GraphVerticesEdges IT-2004Italian web-graph41 Million1.1 Billion UK-2005UK web-graph39 Million0.9 Billion Arabic-2005Arabic web-graph22 Million0.6 Billion FriendsterSocial sub-network10 Million0.6 Billion Erdos-RenyiSynthetic random20 Million2.0 Billion ZigZagSynthetic expander25 Million2.0 Billion

CC Double Greedy Coordination Increase in Coordination % elements failed + blocked Only 0.01% needs blocking!

Concurrency Ctrl. Runtime and Strong-Scaling Sequential

Conclusion Always fastNear optimal Coordination Free Double Greedy Usually fastAlways optimal Concurrency Control Double Greedy Always slowAlways optimal Sequential Double Greedy ScalabilityApproximation Paper @ NIPS 2014: Parallel Double Greedy Submodular Maximization.

BACKUP SLIDES

Concurrency Control Theorem: serializable. preserves optimal approximation bound ½OPT. Lemma: bounded overhead. Coordination Free Lemma: approximation bound ½OPT - error Correctness Concurren cy no overhead Correctness Concurren cy set cover with costs: sparse maxcut: from same dependencies (uncertainty region) set cover with costs: sparse maxcut: Set ASet B w D E z x y x y w uu τ

Adversarial Setting Processing order Overlapping covers  Increased coordination

Adversarial Setting – Ring Set Cover Faster Larger Error Always fast Possibly slow Always optimal Possibly wrongCoord Free Conc Ctrl

Download ppt "Parallel Double Greedy Submodular Maxmization Xinghao Pan, Stefanie Jegelka, Joseph Gonzalez, Joseph Bradley, Michael I. Jordan."

Similar presentations