Download presentation

Presentation is loading. Please wait.

Published byAnastasia Gibson Modified over 2 years ago

1
Scenario Trees and Metaheuristics for Stochastic Inventory Routing Problems DOMinant Workshop, Molde, Norway, September 21.-22., 2009 Lars Magnus Hvattum Norwegian University of Science and Technology, Trondheim, Norway Arne Løkketangen Molde University College, Molde, Norway Gilbert Laporte HEC, Montréal, Canada

2
2 Outline of the presentation

3
3 Inventory Routing Problems extend the VRP, and look at a larger part of a supply chainSupplierCustomer VRP IRP

4
4 The dynamic aspect of inventory routing can be handled in various ways 1) Single Period models 2) Multi-Period models 3) Infinite horizon models (...) Kleywegt, Nori, and Savelsbergh (2002, 2004) Adelman (2004)

5
5 The problem is modelled as a Markov Decision Process, where each epoch corresponds to a day

6
6

7
7 The infinite horizon discounted reward Markov Decision Problem can only be solved for tiny instances A large number of states: all possible inventory levels A large number of actions: all possible delivery patterns A large number of transitions: all possible demand realizations

8
8 Previous solution methods for these SIRPs are based on approximating the value functions Standard algorithms for MDPs can solve some instances: 4 customers,1 vehicle, and 9 inventory levels 5 customers, 5 vehicles, 5 inventory levels, but only direct delivery Kleywegt et al. propose approximations of the value function for instances with direct delivery or at most 3 deliveries per route Adelman proposes approximations of the value function for instances with an unlimited number of vehicles

9
9 To simplify, we only look at finite scenario trees to generate a stochastic policy The infinite horizon is approximated by a finite tree The large number of transitions is approximated by sampled realizations The action space remains unchanged State transitions are based on current inventory level, delivered quantities and sampled demand realizations

10
10 The problem of finding optimal actions conditional on the sampled tree is formalized as an integer program

11
11 We have examined three methods to solve the scenario tree problem (STP) CPLEX Solve IP Solve MIP, where integrality constraints are kept only for root node GRASP Construct solutions to the STP in a randomized adaptive fashion, but with added learning mechanisms Progressive Hedging Algorithm Decompose problem over scenarios, solve each scenario using modified GRASP Gradually enforce an implementable solution by penalties, including a quadratic term

12
12 u = 0,2,0,0 y = 0,0,0,0 d = 1,2,2,1d = 1,1,2,2 y = 0,0,0,0 u = 0,0,0,0u = 0,1,0,0 In GRASP, we start with a solution without any deliveries, then add more deliveries if profitable (...) No deliveries scheduled in any node

13
13 u = 0,2,0,0 y = 0,0,2,0 d = 1,2,2,1d = 1,1,2,2 y = 0,0,0,0 u = 0,0,0,0u = 0,1,0,0 In GRASP, we start with a solution without any deliveries, then add more deliveries if profitable (...) First iteration: add 2 units of delivery to customer 3 using vehicle 1 in node 1 First iteration: add 2 units of delivery to customer 3 using vehicle 1 in node 1 No increase in inventory, but reduction in stock- outs

14
14 u = 0,2,0,0 y = 0,0,2,2 d = 1,2,2,1d = 1,1,2,2 y = 0,0,0,0 u = 0,0,0,1u = 0,1,0,0 In GRASP, we start with a solution without any deliveries, then add more deliveries if profitable (...) Second iteration: add 2 units of delivery to customer 4 using vehicle 1 in node 1 Increase in inventory in node 3 for customer 4

15
15 u = 0,2,0,0 y = 0,0,2,2 d = 1,2,2,1d = 1,1,2,2 y = 0,0,2,0y = 0,0,0,0 u = 0,0,0,1u = 0,1,0,0 In GRASP, we start with a solution without any deliveries, then add more deliveries if profitable (...) Continue making insertions: y additional units to customer i using vehicle k in node v Stop when no profitable, feasible insertions can be made

16
16 Several variations of the GRASP are examined, to find a robust version that can handle different types of instances Restricted Candidate List (RCL) based on either value or rank [former is more robust] Size of RCL is controlled by a parameter that is adjusted dynamically [more robust than a fixed value] Build solution node by node (recursively) or all nodes simultaneously [former is much faster] Use learning based on analysing completed solutions to find potential improvements [increases robustness of building solution node by node]

17
17 The progressive hedging algorithm is based on decomposing the problem over scenarios

18
18 Single scenarios are solved separately, until implementability is enforced through penalties Each scenario is solved using GRASP Objective function is modified to penalize deviations from an averaged solution:

19
19 Several variations of the progressive hedging algorithm are examined, to find a robust version that can handle different types of instances A progressive hedging heuristic is employed to generated feasible solutions to the original scenario tree problem [ensures that good feasible solutions are found even if the method has insufficient time to converge] Penalty parameter is updated dynamically [better balance of progress towards an implementable solution] Use multiple penalty parameters and weights [better than using a single parameter] Use intermediate heuristic solutions for guiding the search [does not work, quicker convergence but to worse solutions] Lock variables for which concensus seems to have been reached [does not work, induces cycling behavior?]

20
20 The search of the resulting progressive hedging algorithm has a fairly similar behavior across instances Left: the averaged solution gives a heuristic upper bound, and the progressive hedging heuristic gives actual lower bounds Right: we can measure the distances in the solution space and the parameter space between iterations, as well as the dynamic penalty parameter

21
21 There are different ways of controlling the computational effort used by the methods Increasing the effort gives improved results Right: profit as a function of the number of GRASP iterations per epoch

22
22 The size of the scenario trees is crucial both for the computational time and the simulation results Increasing the size of the scenario trees increases the computational effort as well as the profits observed

23
23 Several scenario tree problems were studied separately to study the GRASP, the PHA, and the other methods NameLP(-STE)DSExactIP4MIPTDGANG PHA: TDG PHA: ANG STP 01720.2-651.0687.1691.0690.5 692.7693.0 STP 04798.2-724.0753.7756.7757.1757.2754.2755.3 STP 05157626.0---73652.9151958.0151378.0146832.0139283.0 STP 07157352.0--126497.0148095.0151960.0151458.0 143851.0 STP 11-266.8-617.7----473.3-473.4-478.7-461.9 STP 15-420.7-869.4----793.3-772.9-763.1-748.9 STP 16--984.7----938.1-909.4-933.3-925.0 STP 21-270.2-623.9----479.7-478.7-487.5-470.4 STP 26-113.3-909.2----563.4-514.0-518.8-536.7 STP 3167452.7---62012.162678.462620.663283.663336.2 STP 414570.6-4137.8-4195.14111.64181.34116.04161.1 STP 42------160.589.5-900.1-1278.2 STP 431563.0-----1155.3-1149.9-1119.1-1683.9 STP 5133309.4--32537.7 32365.2 STP 5222268.1--21740.9 21433.7 21411.7 STP 551375.0-1071.91232.61213.51229.9 1231.71232.6

24
24 Simulations are run over many epochs (600), and quick heuristics for the scenario tree problems must be selected Our methods: no initial time required, but should allocate some time for the daily problem (time consuming when evaluating, but ok in practice) Other methods: high effort initially (days), but fairly quick for the daily problem (ok when evaluating, practice requires a stable situation)

25
25 GRASP is quicker than PHA, but PHA is better on some instances (produces solutions with different structure)

26
26 Some of the observations made during the work have potential for leading to future research 1) Scenario trees are used to represent the stochastic and dynamic aspects: how to incorporate these in the most efficient way? 2) Scenario tree problems must be solved to generate decisions: how is this best done?

27
27 Scenario trees are not often used in (meta-) heuristics, and several questions remain as to how they should be generated We generate them using random sampling to cover a specified tree structure The size of the tree is determined by specifying the branching factors for each level of the tree

28
28 Can we save computational time by using a clever tree structure? We use lower branching factors lower in the tree (more important to represent stochasticity that is close in time?) We should vary the depth/width of the tree based on the instance solved? We use the same length for every scenario We had to limit the size of the tree to be able to evaluate the methods with simulations

29
29 We can improve results for a given tree size by sampling differently? We also tried a moment matching method, but a requirement of the implementation was that for a single parent node, the number of children must be at least the number of customers (distributions) Research question: can we determine a suitable objective function to be used by a metaheuristic that constructs scenario trees? We use random sampling: by using larger trees we get closer to the true distribution

30
30 We have tested GRASP and PHA for solving the scenario tree problems Local search based methods frequently perform better and faster than GRASP Problem: difficult to find local moves in a scenario tree, as the interconnectedness of decisions creates feasibility issues (how to find a suitable move evaluation? how to guide the search back to feasible space if allowing capacity/inventory violations?)

31
31 One idea is to hybridize local search and construction heuristics Solution: Do local search moves only in this part of the solution (representation) To evaluate each move, this part of the solution must be completed using a construction heuristic (Root node) (Rest of tree)

32
32 Concluding remarks Stochastic and dynamic problems may become increasingly important (with better technology and access to data) The stochastic inventory routing problem is an interesting playground for testing how one can deal with stochastic and dynamic problems Using scenario trees to represent stochasticity is relatively untested in combination with metaheuristics Several directions for future research have been found but not yet pursued

Similar presentations

Presentation is loading. Please wait....

OK

Partially Observable Markov Decision Process (POMDP)

Partially Observable Markov Decision Process (POMDP)

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google