Download presentation

Presentation is loading. Please wait.

Published byHailey Tennies Modified over 2 years ago

1
Primal Dual Combinatorial Algorithms Qihui Zhu May 11, 2009

2
Outline Packing and covering Primal and dual problems Online prediction using feedback Primal-dual combinatorial algorithm Applications and extensions

3
Packing Problem (0-1 Knapsack) n objects with weights W i and prices p i A knapsack with capacity c Pack objects maximizing the total price without exceeding the capacity

4
Packing Problem (0-1 Knapsack) n objects with weights W i and prices p i A knapsack with capacity c Pack objects maximizing the total price without exceeding the capacity Knapsack IP

5
Packing Problem (General) n objects with weights W ij and prices p i m constraints with capacity c j to fit Pack objects maximizing the total price without exceeding the capacities Packing IP

6
Covering Problem n objects to cover at least p i times m sets with costs c j cover each covers W ji objects Cover all objects with the minimal cost

7
Covering Problem n objects to cover at least p i times m sets with costs c j cover each covers W ji objects Cover all objects with the minimal cost Covering IP

8
Fractional Packing Original form

9
Fractional Packing Matrix form Original form

10
Fractional Packing and Covering Are Dual PackingCovering

11
Fractional Packing and Covering Are Dual PackingCovering Covering: best upper bound on packing Packing: best lower bound on covering

12
Decision Version Given constraint set such that For packing: (PST 95) Feasibility Binary search for optimization

13
Flipped Sides of the Same Coin Yes certificateNo certificate Think of the game of twenty questions...

14
Key Idea #1: Need Primal-Dual Algorithm Yes certificateNo certificate Primal: Dual:

15
How to Generate the Certificates? Randomly guessing...

16
How to Generate the Certificates? Randomly guessing... Primal and Dual need to communicate

17
From Dual to Primal Efficient combinatorial algorithm

18
From Dual to Primal Given dual estimate and constraint set Let, find such that Oracle Efficient combinatorial algorithm

19
From Dual to Primal Given dual estimate and constraint set Let, find such that Oracle For packing, essentially ignore all capacity constraints Reduce to sorting over ! Complexity:

20
From Primal to Dual Combination of hyperplanes

21
From Primal to Dual Combination of hyperplanes Tilt the hyperplanes using feedback,..., etc. One step not enough! Getting complicated over iterations... Online Prediction

22
Experts predicting some uncertain event Expert Event

23
Online Prediction Experts predicting some uncertain event Expert Event Value Gain some value from the world (adversarial)

24
Online Prediction Experts predicting some uncertain event Expert Event Value Gain some value from the world (adversarial) Weight Linearly combine by weights

25
Online Prediction Experts predicting some uncertain event Expert Event Long term value over time Time Value Gain some value from the world (adversarial) Weight Linearly combine by weights

26
A Simplified Case Expert 1 Expert 2 Combined Value Average playoff Regret 1 0 10 1 0 1 history ??? ? Only 2 experts 0 1 01 0 1 0 ????? ? ? Value

27
0.9 0 00 0 0 0 Strategy I Expert 1 Expert 2 Combined 0.9 0 10 1 0 1 10101 Take the best from history? 0.1 1 01 0 1 0 10101 1 0 1 0 1 0.5

28
0 1 02/3 1/3 2/3 Strategy II Expert 1 Expert 2 Combined 1 1 01 1 0 1 Linearly weighted by the cumulative values? 0 0 10 0 1 0 0 1 0 1 0.5 112/3 100 1/3

29
0 1 01 1 0 1 Strategy III Expert 1 Expert 2 Combined 1 1 01 1 0 1 Exponentially weighted by the cumulative values! 0 0 10 0 1 0 0 1 0 1 0.5 11 1-ε 100 ε εε

30
Key Idea #2: Need Multiplicative Feedback Yes certificateNo certificate Primal: Dual:

31
From Primal to Dual: Continued Multiplicative Weight Update (MW) Let be the current state at step and be the feedback Combination of hyperplanes

32
Regret Bound Theorem (LW94) Suppose predictions of experts have value. Each time the predictions are combined by weights, where. Update weights using. After time

33
Regret Bound Theorem (LW94) Suppose predictions of experts have value. Each time the predictions are combined by weights, where. Update weights using. After time Proof. Consider the potential function Show that it is dominated by the best expert.

34
Primal-Dual Algorithms Primal Dual Multiplicative Weight Update Oracle

35
Algorithm Initialize:, Repeat Set Oracle If then Return infeasible, output Set Multiplicative Weight Update Until Return feasible solution Primal-Dual Combinatorial Algorithm

36
Analysis Initialize:, Repeat Set Oracle If then Return infeasible, output Set Multiplicative Weight Update Until Return feasible solution Primal-Dual Combinatorial Algorithm Oracle: if failed, infeasible

37
Analysis Initialize:, Repeat Set Oracle If then Return infeasible, output Set Multiplicative Weight Update Until Return feasible solution Primal-Dual Combinatorial Algorithm Oracle: if failed, infeasible Regret bound: Width: Feasible solution! Complexity depends on

38
Applications Fractional packing and covering Multicommodity flow Held-Karp Bound for TSP Semidefinite programming (SDP) relaxation General convex programming Boosting Matrix game

39
Applications Fractional packing and covering Multicommodity flow Held-Karp Bound for TSP Semidefinite programming (SDP) relaxation General convex programming Boosting Matrix game

40
Multicommodity Flow Objective: maximizing total flow while respecting capacities

41
Multicommodity Flow Objective: maximizing total flow while respecting capacities Packing paths Update: route some flow along shortest path Multicommodity Flow Congestion

42
Initialize:, Repeat Set Oracle If then Return infeasible, output Set Multiplicative Weight Update Until Return feasible solution Primal-Dual Combinatorial Algorithm Multicommodity Flow Oracle : shortest path keep pushing flows through P : polytope of graph flows Augment length (GK 98)

43
SDP Relaxation Max Cut (GW 95) Better bounds on Max Cut, Sparsest Cut, Max-2Sat, etc. Finding good geometric embedding by SDP + rounding Interior point method not scale- Sparsest Cut (ARV 04) Primal SDPDual SDP

44
Initialize:, Repeat Set Oracle If then Return infeasible, output Set Multiplicative Weight Update Until Return feasible solution Primal-Dual Combinatorial Algorithm Primal SDP P : set of semidefinite matrices Oracle : compute eigenvectors!

45
Initialize:, Repeat Set Oracle If then Return infeasible, output Set Multiplicative Weight Update Until Return feasible solution Primal-Dual Combinatorial Algorithm Dual SDP P & Oracle : same as packing Multiplicative Weight Update Use matrix exponential !

46
Initialize:, Repeat Set Oracle If then Return infeasible, output Set Multiplicative Weight Update Until Return feasible solution Primal-Dual Combinatorial Algorithm Weighted Majority Algorithm/Boosting Events/Classification results: not controlled by us! Error on training examples

47
Summary Combinatorial subroutines

48
Summary Fast algorithms!

49
Conclusion A computational paradigm –Applied to numerous problems –Cover at least convex problems Fast approximation algorithms –Trade accuracy for time: O(1/ε) –Fast combinatorial algorithms for oracle (without solving LP!) Flexibility for algorithm design –Multiplicative weight update: a principle way to use feedback –Free to separate and combine constraints Width management is important –Most algorithms polynomial to width –Separate high width constraints: nested primal-dual –Decompose constraint set

50
Thank you! Questions …

51
Regret Bound Theorem (LW94) Suppose predictions of experts have value. Each time the predictions are combined by weights, where. Update weights using. After time Proof. Consider the potential function Show that it is dominated by the best expert.

52
Update rule

53
Exp function Width

54
Update rule Exp function Width

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google