Optimal resampling using machine learning Jesse McCrosky

Outline Introduction Policies  Costs  States Methodology  Data collection  Learning Results  Rho values  Simulation comparison

Introduction SERP has a parameter, rho, to control amount of resampling  Using correct value can substantially improve performance What is correct value?  Depends on filter state

Policies A policy is a mapping between states and actions  State is state of filter – discussed later  Actions are rho values to use An optimal policy minimizes the expected value of some cost function

Costs The one-step cost is simply:  Could also consider computation time and variance of estimate So for one iteration from time k to k+1, the optimal policy is: But this policy is greedy  Might result in a very low cost at time k+1, but result in very high costs later

Time Horizon Cost The one-step policy just defined is optimal for the last iteration of the filter The optimal i th step policy is:

Time Horizon Cost (continued) Time horizon cost combines cost of current iteration, plus next i iterations  Future iterations are discounted For general use, we want infinite time horizon: This should converge (hopefully quickly)

States Filter state is aggregate of discretized value of six state elements  Expected number of targets  Variance of expected number of targets  Change in median weight since last iteration  Boxplot state (3 elements)  Would also like to use variance of target position estimate, but difficult to define and calculate with unknown number of targets

Boxplot Boxplot state consists of three elements:  u1 – u4, u1 – u3, and u1 – u2 Where u1 is the highest weight u2 is the 75% median weight u3 is the 50% median weight u4 is the 25% median weight

Methedology Two approaches  Dynamic programming  Artificial learning Both cases involve trying various rho values in various states many times and finding lowest average cost

Dynamic Programming Create artificial particle set for each possible state and try each rho value Advantage: guaranteed data for every state Disadvantage: particle sets are artificial and may be unrealistic or non-representative Not used in paper  Outside area of interest for conference

Artificial Learning Evolve filter naturally with real signal and record costs in state vs. rho table Then use neural network to approximate optimal rho for states not encountered in training Advantages: uses “real” data, neural network can compensate for some bad data Disadvantages: slow to train, depends on quality of neural network

Data Collection The Rho Optimizer will generate a signal and attempt to filter it Each iteration, record the filter state, rho value, and cost Other details: i th step optimizer, choosing rho value, epochs

Learning Optimizer uses collected data to train a neural network After training the network will output a (approximately) correct value for any states in the data and a (hopefully) correct value for other states

Results Currently have one step results only 2000 epochs of 200 iterations each Each state variable discretized into 10 cells, except sigma-numtargets which has 5 10 possible rhoint values, from 1000 to

Graphs Using data collection stage data only, no neural network Optimal rho value on y-axis vs. discrete index of state component on x-axis Optimal rho value is average of optimal rhos for all states which match values in graph’s state component  Graph are biased by states encountered in simulation Because of bias and flattening, graphs are for novelty purposes only

Boxplot - 1

Boxplot - 2

Boxplot - 3

Delta Median Weight Note: index 2 corresponds with deltamedian = 0

Expected number of targets

Variance of expected number of targets

Graphs - Conclusions Correlation looks good in some elements Bad graphs maybe OK, for example, boxplot may look better if plotted in 4 dimensions Some surprises, does expected number of targets not matter? Real test of results will be simulation comparison between optimal and constant rhos

Simulation No simulations yet  Neural network still training as we speak Next week – Nikki will present her PHD simulations and I will present results of comparative simulations