1 Random Disambiguation Paths Al Aksakalli In Collaboration with Carey Priebe & Donniell Fishkind Department of Applied Mathematics and Statistics Johns Hopkins University Adaptive Sensing MURI Workshop June 28, 2006 Duke University

2 1)Problem Description 2)Markov Decision Process Formulation 3)Simulated Risk Disambiguation Protocol 4)Computational Experiments 5)Ongoing Research 6)Summary and Conclusions Outline:

3 Spatial arrangement of detections: true detections, false detections Problem Description:

Spatial arrangement of detections: true detections, false detections Assume for all that is the probability that Problem Description: We only see

5 Given start and destination start t destination s Problem Description:

6 About each detection there is a hazard region, an open disk of fixed radius s t Given start and destination Problem Description:

s t ?? About each detection there is a hazard region, an open disk of fixed radius Given start and destination We seek a continuous curve from to in of shortest achievable arclength Problem Description:

s t About each detection there is a hazard region, an open disk of fixed radius Given start and destination …and we assume the ability to disambiguate detections from the boundary of their hazard regions. We seek a continuous curve from to in of shortest achievable arclength Problem Description:

s t About each detection there is a hazard region, an open disk of fixed radius Given start and destination …and we assume the ability to disambiguate detections from the boundary of their hazard regions. true We seek a continuous curve from to in of shortest achievable arclength Problem Description:

s t About each detection there is a hazard region, an open disk of fixed radius Given start and destination …and we assume the ability to disambiguate detections from the boundary of their hazard regions. …or false We seek a continuous curve from to in of shortest achievable arclength Problem Description:

s t About each detection there is a hazard region, an open disk of fixed radius Given start and destination …and we assume the ability to disambiguate detections from the boundary of their hazard regions. the rest of the transversal… We seek a continuous curve from to in of shortest achievable arclength Problem Description:

12 Definition: A disambiguation protocol is a function # disambiguations allowed cost per disambiguation which detection disambiguated next… …and where the disambiguation performed

13 Example 1: Protocol gives rise to the RDP Length=707.97, prob=.89670Length= , prob=.10330

14 Example 2: Protocol gives rise to the RDP (superimposed composite)

15 Random Disambiguation Paths (RDP) Problem: Given, find protocol of minimum.

16 Related work: Canadian Traveller Problem (CTP): Graph theoretic RDP Given a finite graph – edges with specific probabilities of being traversable, and a starting and a destination vertex – each edge’s status is revealed only when one of the end points is visited: objective is to minimize expected traversal length Shown to be #P-hard

17 Markov Decision Process (MDP) formulation: Let be the information vector keeping track of the decision maker’s current knowledge; be the set of all possible disambiguation points RDP Problem can be cast as a K-stage finite horizon MDP with States: Actions: where v is a disambiguation point and i is a hazard region index Rewards: the negative of the shortest path distance between the state vertex and the action vertex minus c, if not going to d - d is an absorbative state for which there is a one-time and very large reward for entering Transitions: governed by ‘s

18 Simulated Risk Protocol: For purpose of deciding next disambiguation point, we pretend that ambiguous disks are riskily traversable… traversal? ? ? ? ?

19 is the surprise length of, which is the negative logarithm of the probability that is traversable in actuality. Risk Simulation Protocol: For purpose of deciding next disambiguation point, we pretend that ambiguous disks are riskily traversable… traversal? ? ? ? ? is the usual Euclidean length of.

20 Given undesirability function (henceforth, monotonically non-decreasing in its arguments) and, say,

21 Given undesirability function (henceforth, monotonically non-decreasing in its arguments) and, say, Definition: The simulated risk protocol is defined as dictating that the next disambiguation be at the first ambiguous point of. traversal? ? ? ? ?

22 Given undesirability function (henceforth, monotonically non-decreasing in its arguments) and, say, Definition: The simulated risk protocol is defined as dictating that the next disambiguation be at the first ambiguous point of. traversal? ? ? ? ? How to proceed once this disambiguation is performed: update and, decrement, and set the new s to be y.

23 How to navigate in this continuous setting: The Tangent Arc Graph (TAG) is the superimposition/subdivision of all visibility graphs generated by all subsets of disks.  For any undesirability function, is an path in TAG !

24 Linear undesirability functions: Because of the efficiency in their realization, we will consider simulated risk protocols generated by linear undesirability functions for a chosen parameter. As a further shorthand, denote such a protocol by.

25 How (during the simulation of risk phase) can be affected by :

26 How (during the simulation of risk phase) can be affected by :

27 How (during the simulation of risk phase) can be affected by :

28 How (during the simulation of risk phase) can be affected by :

29 How (during the simulation of risk phase) can be affected by :

30 Example 1: Protocol gives rise to the RDP Length=707.97, prob=.89670Length= , prob=.10330

31 Example 2: Protocol gives rise to the RDP (superimposed composite)

32 Lattice Discretization: Discretization via a subgraph of the integer lattice with unit edge lengths:

33 Example: Adapting the simulated risk protocol to lattice discretization:

34 A 40 by 20 integer lattice is used Each hazard region is a disk with radius 5.5 Disk centers sampled from a uniform distribution of integers in ‘s sampled from uniform distribution on (0,1) Cost of disambiguation is taken as 1.5 For each N, K combination, 50 different instances were sampled Optimal solutions found by solving the MDP model via value iteration Computational experiments:

35 Illustration with N=7, K=1: Expected length:

36 Runtime to find overall optimal (SR-RDP runtime negligible) Comparison of optimal versus simulated risk: Simulated risk found the optimal solution 74% of the time Overall mean percentage error of simulated risk solutions was less than 1% For N=7, K=3; VI took more than an hour for N=10, K=1; VI did not run due to insufficient memory

37 Ongoing Research: Pruning State Space via AO* Implemented an enhanced version of AO-star algorithm Preliminary results suggest up to 99% of the state space can be pruned N=15, K=2 can be solved under 15 mins! Not practical for K>2: N=15, K=3 takes 10.5 hours!!! Simulated Risk protocol still seems to perform well

38 Example: Enhanced AO* with N=15, K=2

39 Ongoing Research: Multiple sensors & Neutralization Deployment of multiple sensors with different accuracy rates & ranges at different costs Also consider a limited neutralization capability Develop and solve corresponding Partially Observable Markov Decision Process (POMDP) models

40 Summary and Conclusions RDP is an important, yet hard mine-countermeasures problem Obtaining optimal solutions presently not feasible for realistic values of N and K Simulated risk protocol is a sub-optimal yet efficient algorithm that performed well in computational experiments

41 Q & A