We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byJalynn Allington
Modified about 1 year ago
MIT and James Orlin © 2003 1 Dynamic Programming 1 –Recursion –Principle of Optimality
MIT and James Orlin © 2003 2 Dynamic Programming Transforms a complex optimization problem into a sequence of simpler ones. Usually begins at the end and works backwards to the beginning. Can handle a wide range of problems Relies on recursion, and on the principle of optimality Developed by Richard Bellman
MIT and James Orlin © 2003 3 Recursion example There are 11 people in a room. How many ways can one select exactly 6 of them? Let f(n,k) denote the number of subgroups of size k out of n people. We want f(11,6) 1234567891011 The number of subgroups containing “1” is f(10,5). The number of subgroups not containing “1” is f(10,6).
MIT and James Orlin © 2003 4 f(n,k) = f(n-1,k-1) + f(n-1, k) f(n,n)=f(n,0)=1 f(0,0) f(1,0) f(2,0) f(3,0) f(4,0) f(5,0) f(6,0) f(7,0) f(8,0) f(1,1) f(2,1)f(2,2) f(3,1)f(3,2)f(3,3) f(4,1)f(4,2)f(4,3)f(4,4) f(5,1)f(5,2)f(5,3)f(5,4)f(5,5) f(6,1)f(6,2)f(6,3)f(6,4)f(6,5)f(6,6) f(7,1)f(7,2)f(7,3)f(7,4)f(7,5)f(7,6)f(7,7) f(8,1)f(8,2)f(8,3)f(8,4)f(8,5)f(8,6)f(8,7)f(8,8) 1 1 1 1 1 1 1 1 1 1 21 331 4641 510 51 615201561 72135 2171 8285670562881 Reduces each problem to previously solved problem(s).
MIT and James Orlin © 2003 5 A warm-up example Suppose that there are 20 matches on a table, and the person who picks up the last match wins. At each alternating turn, my opponent or I can pick up 1 or 2 matches. Assuming that I go first, how can I be sure of winning the game? (Play with partner, and then discuss).
MIT and James Orlin © 2003 6 20 Matches Choose a person to go first. Alternate in eliminating 1 or 2 matches. The person who takes the last match wins.
MIT and James Orlin © 2003 7 20 Matches strategy What is the optimal strategy?
MIT and James Orlin © 2003 8 An Example for Dynamic Programming: 20 matches Version 2 Choose a person to go first. Alternate in eliminating 1 or 2 or 6 matches. The person who takes the last match wins.
MIT and James Orlin © 2003 9 20 Matches
MIT and James Orlin © 2003 10 Winning and losing positions (in the game in 20 matches version 2) Suppose you have k matches left. Identify whether you have a Winning strategy (assuming your opponent plays optimally) or whether you will Lose. 12345678910 WWL 11121314151617181920
MIT and James Orlin © 2003 11 Determining the strategy using DP n = number of matches left f(n) = W if you can force a win at n matches. f(n) = L otherwise Determine f(1), f(2), …, f(6) For n > 6, determine f(n) from previously solved problems. f(n) = W if f(n-1) = L or f(n-2) = L or f(n-6) = L; f(n) = L if f(n-1) = W and f(n-2) = W and f(n-6) = W.
MIT and James Orlin © 2003 12 Determining the strategy using DP n = number of matches left (n is the state/stage) f(n) = W if you can force a win at n matches. f(n) = L otherwise f(n) = optimal value function. Determine f(1), f(2), …, f(6) (Boundary conditions) For n > 6, determine f(n) using a dynamic programming recursion f(n) = W if f(n-1) = L or f(n-2) = L or f(n-6) = L; f(n) = L if f(n-1) = W and f(n-2) = W and f(n-6) = W.
MIT and James Orlin © 2003 13 12345678910 11121314151617181920 21222324252627282930 31323334353637383940 41424344454647484950 12345678910 11121314151617181920 21222324252627282930 31323334353637383940 41424344454647484950 Running the Recursion Who sees the pattern determining the losing hands?
MIT and James Orlin © 2003 14 1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950 The same table
MIT and James Orlin © 2003 15 Examples Suppose there are 30 matches on a table. I begin by picking up 1, 2, or 3 matches. Then my opponent must pick up 1, 2, or 3 matches. We continue in this fashion until the last match is picked up. The player who picks up the last match is the loser. How can I (the first player) be sure of winning the game?
MIT and James Orlin © 2003 16 Examples Suppose there are 40 matches on a table. I begin by picking up 1, 2, 3, or 4 matches. Then my opponent must pick up 1, 2, 3, or 4 matches. We continue until the last match is picked up. The player who picks up the last match is the loser. Can I be sure of victory? If so, how?
MIT and James Orlin © 2003 17 Examples I have a 9-oz cup and a 4-oz cup. My mother has ordered me to bring home exactly 6 oz of milk. How can I accomplish this goal? We have 21 coins and are told that one is heavier than any of the other coins. How many weighings on a balance will it take to find the heaviest coin?
MIT and James Orlin © 2003 18 Finding the shortest path from node s to all other nodes in an acyclic network. s
MIT and James Orlin © 2003 19 Shortest Paths Find the shortest path from node 1 to each other node, in order. –What is the shortest path from node 1 to node 2, 3, 4, … ? –Determine these values one at a time. Shortest path
MIT and James Orlin © 2003 20 Finding the shortest path to node t from all other nodes in an acyclic network. t
MIT and James Orlin © 2003 21 Shortest Paths Find the shortest path to node 19 from each node in order. –What is the shortest path to node 19 from node 18, 17, 16….? –Determine these values one at a time. Shortest path
MIT and James Orlin © 2003 22 Dynamic Programming in General Break up a complex decision problem into a sequence of smaller decision subproblems. Stages: one solves decision problems one “stage” at a time. Stages often can be thought of as “time” in most instances. –Not every DP has stages –The previous shortest path problem has 7 stages –The match problem does not have stages.
MIT and James Orlin © 2003 23 Dynamic Programming in General States: The smaller decision subproblems are often expressed in a very compact manner. The description of the smaller subproblems is often referred to as the state. –match problem: “state” is the number of matches left –shortest path problem: “state” is the node At each state-stage, there are one or more decisions. The DP recursion determines the best decision. –match problem: how many matches to remove –shortest path example: which arc to choose
MIT and James Orlin © 2003 24 Principle of Optimality Any optimal policy has the property that whatever the current state and decision, the remaining decisions must constitute an optimal policy with regard to the state resulting from the current decision. (As formulated by Richard Bellman.) Whatever node j is selected, the remaining path from j to the end is the shortest path starting at j.
MIT and James Orlin © 2003 25 Forwards recursion Start with stage 1 and determine values for larger stages using recursion In the match examples, determine f(n) from f(k) for k < n.
MIT and James Orlin © 2003 26 Backwards Recursion Boundary conditions: optimum values for states at the last stage Determine optimum values for other stages using backwards recursion
MIT and James Orlin © 2003 27 Summary for Dynamic Programming Recursion Principle of optimality states and stages and decisions useful in a wide range of situations –games –shortest paths –more will be presented in next lecture.
Tuesday, April 30 Dynamic Programming – Recursion – Principle of Optimality Handouts: Lecture Notes.
MIT and James Orlin © Dynamic Programming 2 –Review –More examples.
Thursday, May 2 Dynamic Programming – Review – More examples Handouts: Lecture Notes.
MIT and James Orlin © Stochastic Dynamic Programming –Review –DP with probabilities.
D Nagesh Kumar, IIScOptimization Methods: M5L2 1 Dynamic Programming Recursive Equations.
Introduction and Preliminaries D Nagesh Kumar, IISc Water Resources Planning and Management: M4L1 Dynamic Programming and Applications.
Optimization of thermal processes2007/2008 Optimization of thermal processes Maciej Marek Czestochowa University of Technology Institute of Thermal Machinery.
1 1 © 2003 Thomson /South-Western Slide Slides Prepared by JOHN S. LOUCKS St. Edward’s University.
Chapter 18 Deterministic Dynamic Programming to accompany Operations Research: Applications and Algorithms 4th edition by Wayne L. Winston Copyright (c)
1 1 Slide © 2000 South-Western College Publishing/ITP Slides Prepared by JOHN LOUCKS.
Dynamic Programming. Well known algorithm design techniques:. –Divide-and-conquer algorithms Another strategy for designing algorithms is dynamic programming.
CSCE350 Algorithms and Data Structure Lecture 17 Jianjun Hu Department of Computer Science and Engineering University of South Carolina
PROBLEM-SOLVING TECHNIQUES Rocky K. C. Chang November 10, 2015.
6 Sums of Games.. Lecture6 Admin Plagiarism forms Answer football cup final problem. //Daisy Problem hints e.g. try a smaller size. General problem solving.
MIT and James Orlin © Game Theory 2-person 0-sum (or constant sum) game theory 2-person game theory (e.g., prisoner’s dilemma)
1 Chapter 15-1 : Dynamic Programming I. 2 Divide-and-conquer strategy allows us to solve a big problem by handling only smaller sub-problems Some problems.
22-Jan-14 Dynamic Programming. 2 Counting coins To find the minimum number of US coins to make any amount, the greedy method always works At each step,
Prime Time 1.2: Playing to Win the Factor Game. 1.2 Playing to Win the Factor Game Goals: – Classify numbers as prime or composite – Recognize that some.
Dynamic Programming In this handout A shortest path example Deterministic Dynamic Programming Inventory example Resource allocation example.
AE1APS Algorithmic Problem Solving John Drake.. Previously we introduced matchstick games, using one pile of matches It is possible to have more than.
Deterministic Dynamic Programming. Dynamic programming is a widely-used mathematical technique for solving problems that can be divided into stages.
1 Chapter 9 Dynamic Programming General Problem-Solving strategy Was born in the 1950s (Bellman) Based on successive decomposition Leads to functional.
Algorithms + L. Grewe. Algorithms and Programs Algorithm: a method or a process followed to solve a problem. –A recipe. An algorithm takes the input to.
Algorithm Strategies Nelson Padua-Perez Chau-Wen Tseng Department of Computer Science University of Maryland, College Park.
Dynamic Programming Discrete time frame Multi-stage decision problem Solves backwards.
Time for playing games Form pairs You will get a sheet of paper to play games with You will have 12 minutes to play the games and turn them in.
Greedy Algorithms Analysis of Algorithms1. Greedy Algorithms A greedy algorithm always makes the choice that looks best at the moment –Everyday examples:
AE1APS Algorithmic Problem Solving John Drake. Invariants – Chapter 2 River Crossing – Chapter 3 Logic Puzzles – Chapter 5 Matchstick Games -
Games Game 1 Each pair picks exactly one whole number The lowest unique positive integer wins Game 2: Nim Example: Player A picks 5, Player B picks 6,
Lecture 7: Greedy Algorithms II Shang-Hua Teng. Greedy algorithms A greedy algorithm always makes the choice that looks best at the moment –My everyday.
Types of Algorithms. 2 Algorithm classification Algorithms that use a similar problem-solving approach can be grouped together We’ll talk about a classification.
What are the first five multiples of: a)3 b)4 c)11 d)21 3, 6, 9, 12, 15 4, 8, 12, 16, 20 11, 22, 33, 44, 55 21, 42, 63, 82, 105 A multiple of a number.
Investigation #1 Factors and Products. 1.1 Playing the Factor Game.
Part2 AI as Representation and Search Dr. Bernard Chen Ph.D. University of Central Arkansas Spring 2011.
1 Tardiness Models Contents 1. Moor’s algorithm which gives an optimal schedule with the minimum number of tardy jobs 1 || U j 2. An algorithm which.
Dynamic Programming - DP December 18, 2014 Operation Research -RG 753.
8 -1 Dynamic Programming Fibonacci sequence Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … F i = i if i 1 F i = F i-1 + F i-2 if i 2 Solved.
D Nagesh Kumar, IIScOptimization Methods: M5L1 1 Dynamic Programming Introduction.
Fundamentals of Algorithms MCS - 2 Lecture # 7. Types of Algorithms.
Investigation #1 Factors (Factor Game). 1.1 Playing the Factor Game.
Dynamic Programming Key characteristic of a Dynamic Program: Breaking up a large, unwieldy problem into a series of smaller, more tractable problems. Shortest.
Lecture Coursework 2 AGAIN. Rectangle Game Look at proof of matchsticks A rectangular board is divided into m columns by n rows. The area of the board.
Computer Sciences Department1. Property 1: each node can have up to two successor nodes (children) The predecessor node of a node is called its.
ECE 667 Synthesis & Verification - Technology Mapping (ASIC) 1 ECE 667 Synthesis and Verification of Digital Systems Technology Mapping (standard cells)
PING PONG (Table Tennis) History – Ping pong is of English origin. It was first played on dining room tables. It can be played with 2 or 4 players. To.
15.082J, 6.855J, and ESD.78J September 21, 2010 Eulerian Walks Flow Decomposition and Transformations.
David Luebke 1 8/23/2015 CS 332: Algorithms Greedy Algorithms.
November 10, 2009Introduction to Cognitive Science Lecture 17: Game-Playing Algorithms 1 Decision Trees Many classes of problems can be formalized as search.
David Luebke 1 10/24/2015 CS 332: Algorithms Greedy Algorithms Continued.
Eastern Mediterranean University Department Of Industrial Engineering Deterministic Dynamic Programming presented by: Ziad El Masri Supervisor: Prof. Dr.
© 2017 SlidePlayer.com Inc. All rights reserved.