1. A temporally abstracted Viterbi algorithm (TAV) Shaunak Chatterjee and Stuart Russell University of California, Berkeley July 17, 2011

2. Earth’s history – A timescale view Widely varying timescales are pervasive in data Planning, simulation & state estimation – More efficient if timescale information is cleverly exploited 4.5Ga1Ma10000 yrs 600 yrs 1 yr2 days1 min

3. Where is Shaunak? MondayTuesdayWednesdayThursdayFridaySaturdaySunday Berkeley Barcelona Philadelphia Barcelona Burger Cheese steak PaellaGazpachoTapasGazpacho Images: berkeley.edu, wikipedia, food.com

4. State time trellis t=1t=2t=3t=4t=5 Berkeley Philadelphia Montreal Toronto Barcelona Madrid Paris Marseille t=6

5. The Viterbi algorithm – Viterbi, 1967 1 2 3 3 4 4 4 4 Berkeley Philadelphia Montreal Toronto Barcelona Madrid Paris Marseille t=1t=2t=3t=4t=5 t=6

6. 1 2 3 3 4 4 4 4 3 4 4 4 6 6 8 8 t=1t=2t=3t=4t=5 t=6 Berkeley Philadelphia Montreal Toronto Barcelona Madrid Paris Marseille The Viterbi algorithm – Viterbi, 1967

7. 1 2 3 3 4 4 4 4 3 4 4 4 6 6 8 10 7 15 17 18 15 18 8 t=1t=2t=3t=4t=5 t=6 Berkeley Philadelphia Montreal Toronto Barcelona Madrid Paris Marseille The Viterbi algorithm – Viterbi, 1967

8. 1 2 3 3 4 4 4 4 3 4 4 4 6 6 8 10 7 15 17 18 15 18 8 t=1t=2t=3t=4t=5 t=6 Berkeley Philadelphia Montreal Toronto Barcelona Madrid Paris Marseille 13 15 10 11 13 15 The Viterbi algorithm – Viterbi, 1967

9. 1 2 3 3 4 4 4 4 3 4 4 4 6 6 8 10 7 15 17 18 15 18 8 t=1t=2t=3t=4t=5 t=6 Berkeley Philadelphia Montreal Toronto Barcelona Madrid Paris Marseille 13 15 10 11 13 15 14 15 16 17 11 12 14 15 16 17 18 12 13 15 16 The Viterbi algorithm – Viterbi, 1967

10. 1 2 3 3 4 4 4 4 3 4 4 4 6 6 8 10 7 15 17 18 15 18 8 t=1t=2t=3t=4t=5 t=6 Berkeley Philadelphia Montreal Toronto Barcelona Madrid Paris Marseille 13 15 10 11 13 15 14 15 16 17 11 12 14 15 16 17 18 12 13 15 16 10 The Viterbi algorithm – Viterbi, 1967

11. O(N 2 T) by using dynamic programming – N T possible state sequences Used in signal decoding, speech recognition, parsing and many other applications For large N and T, this cost could be quite prohibitive Every possible transition is considered – In some cases, many of these transitions are very unlikely to feature in the optimal path

12. Abstraction U.S.A. Canada Spain France Europe North America Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille t=1t=2t=3t=4t=5 t=6

13. Abstraction tree Abstractness  Europe Spain Barcelona France Madrid Paris Marseille

14. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

15. CFDP Step 1: Find the most likely sequence in the current state-time trellis

16. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

17. CFDP Step 1: Find the most likely sequence in the current state-time trellis Step 2: Refine along the most likely sequence

18. CFDP Refinement Node-based refinement N.America Europe N.America Spain France Node Refinement

19. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

20. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

21. CFDP Step 1: Find the most likely sequence in the current state-time trellis Step 2: Refine along the most likely sequence Step 3: Go to step 1 if step 2 performed any refinement; else terminate

22. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

23. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

24. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

25. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

26. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

27. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

28. Cost bounds for abstract links Cost of an abstract link should be a lower bound of the link refinements it encapsulates Standard heuristic admissibility argument  Correctness

29. Coarse-to-fine dynamic programming (CFDP) – Raphael, 2001 t = 1t = 2t = 3t = 4t = 5t = 6 Berkeley Philly Montreal Toronto Barcelona Madrid Paris Marseille

30. Analyzing CFDP Great when large portions of the state-time trellis are very unlikely – Leading to fewer refinements An appropriate abstraction hierarchy is required

31. Analyzing CFDP Great when large portions of the state-time trellis are very unlikely – Leading to fewer refinements An appropriate abstraction hierarchy is required Computation complexity – Best case – O(B 2 T (log N) 3 ) B is the branching factor of the hierarchy – Worst case – O(N 4 T 2 )

32. An actual state trajectory JanDec Berkeley San Francisco Stanford Sardinia Venice Milan Interlaken Los Angeles road trip Yosemite road trip India trip Europe trip MaySep

33. Persistence a.k.a. Timescales JanDec Berkeley San Francisco Stanford Sardinia Venice Milan Interlaken Los Angeles road trip Yosemite road trip India trip Europe trip MaySep

34. Persistence and timescales People stay within the same metropolitan area for weeks Change countries in months Continent changes are more rare We do not need to consider transitions from the Bay Area to other continents and countries at every time step

35. A set of really good paths Set 1: All paths within California for the entire month of April Set 2: All paths that visit California and at least one other state in April Cost(Paths April-in-California ) < Cost(Paths April-in-1+-states ) | Paths April-in-California | << | Paths April-in-1+-states | An abstraction scheme which can distinguish between these two sets of paths!

36. Temporally abstract link Each link encapsulates a set of paths at the specified abstraction level over a temporal interval [T 1,T 2 ] Just specifying start and end points is pointless! T1T1 T2T2 N.America Europe

37. Links T1T1 T2T2 Direct links Paths that stay within N. America for the entire interval [T 1,T 2 ] N.America Europe

38. Links N.America T1T1 Europe T2T2 Cross links Paths that start in Europe at T 1 and end in N. America at T 2

39. Links T1T1 T2T2 Re-entry links Paths that start and end in N. America at T 1 and T 2 respectively, but leave N.America at least once in that interval N.America Europe

40. Link Refinement No longer refining nodes! Two types of refinement – Direct links undergo spatial refinement – Cross and re-entry links undergo temporal refinement

41. Spatial Refinement U.S.A. T2T2 T1T1 T2T2 T1T1 Canada N.America Europe N.America Europe

42. Temporal Refinement T2T2 T1T1 T2T2 T1T1 T’ N.America Europe N.America

43. TAV algorithm Identical to CFDP in structure Step 1: Find the most likely sequence in the current state-time trellis Step 2: Refine along the most likely sequence – Link refinement instead of node refinement Step 3: Go to step 1 if step 2 performed any refinement; else terminate

44. TAV example T0 N.America Europe

45. TAV algorithm - example

60. TAV algorithm - demo

61. Score computation Abstractness  Europe Spain Barcelona France Madrid

62. Score computation Abstractness  Europe Spain Barcelona France Madrid

63. Score computation Abstractness  Europe Spain Barcelona France Madrid

64. TAV vs CFDP

65. Design choices Temporal refinement – One link or all links – Splitting point Computing cost bounds – Bound on paths within California in April – Tradeoff between precision and computation cost Abstraction hierarchy – Deep vs shallow

66. Simulation setup TAV works well when the system has a wide range of timescales We set up a DBN with n binary variables – Similar to the continent, country, city example but with more levels The i th variable had a timescale of (1/ε) i

67. Results TAV outperforms CFDP and Viterbi for various values of T, N and ε 1/ε is the timescale gap between hierarchy levels

68. Hierarchy induction Deep hierarchies are better suited for TAV, even when the underlying model has a shallow abstraction hierarchy

69. Comments Much faster than CFDP and Viterbi in a system with multiple timescales The speedup is a function of the range of timescales Not suited for applications without timescales (persistence). In the worst case, TAV is much slower than CFDP and Viterbi

70. Conclusion Efficient inference algorithms can be designed for systems with a wide range of timescales – Conventional algorithms often cannot exploit this extra structure TAV benefits significantly from considering locally constrained trajectories – Using such constrained local search to build a global solution was something that previous DP formulations did not do

71. THANK YOU! On a lighter note: Counter for grad students being currently tracked by TAV: 0