1. University of Virginia Department of Systems & Information Engineering Feature Selection, Prediction, and Change Detection in Terrorist/Insurgency Processes Using a Spatial Point Process Approach Michael D. Porter and Donald E. Brown {mdp2u,brown}@virginia.edu Department of Systems and Information Engineering University of Virginia

2. Department of Systems & Information Engineering 2 AgendaAgenda Problem Statement Approach & Specification Feature Selection (Review) Prediction (Review) Change Detection IED Example /42

3. University of Virginia Department of Systems & Information Engineering 3 Terrorism Against the U.S. Terrorist attacks take many forms –suicide bombings –improvised explosive devices –hostage taking –mortar & rocket attacks –Complex attacks The incident on the right was a suicide bombing at a police station in Iraq that occurred on February 12, 2004 & killed 47 people How can we become proactive in addressing these threats? Hull, Bryson, “100 die in two Iraq suicide bombings,” The Age, February 12, 2004, http://www.theage.com.au/ /42

4. University of Virginia Department of Systems & Information Engineering 4 Modeling Terrorism and Insurgency PREDICTION Proactive in force deployment Awareness of threat locations and levels Should include terrorist decision making CHANGE DETECTION New terrorist group Interventions effects (arrests, patrols, etc) New terrorist strategy/preferences /42

5. University of Virginia Department of Systems & Information Engineering 5 Intelligent Site Selection Assumptions: Terrorists judiciously plan the location of their attacks according to their preferences or perceived utility of that location –Site selection is not made by considering spatial coordinates alone Site selection is made based on the attributes or features of each possible location Capturing these features that are considered by the terrorist will lead to better (more accurate) models /42

6. University of Virginia Department of Systems & Information Engineering 6 Criminal Ecological Theory In criminology, ecological theories seek to describe the motivations and acts of crime based on the general features of one’s environment –Environment of criminal –Environment of crime scene This points to the explanation that there are some features of the location of crimes which are important to criminal decision making We extend this thought to terrorist decision making /42

7. University of Virginia Department of Systems & Information Engineering 7 Feature Set To successfully model the terrorist attacks, we should attempt to model their decision making process or preferences for attack locations Thus we include features that are thought to influence the terrorist site selection process (or that are associated (correlated) with such features) in our models Since we usually don’t know the terrorist’s preferences we must discover these from previous attack locations –Observe past attack locations and associated feature values for that location Examples of possible features –Census (Socio-economic) –Proximity (Distance to landmarks or structures) –Military or Police Patrols (times and locations) /42

8. University of Virginia Department of Systems & Information Engineering 8 Point Processes A stochastic model governing the location and number of events in some set X [Cressie, 1993] A point process P = {x i : x i  X }, where the x i are the events and X is the set over which the point process is defined Specified by where  - mean measure and  - intensity function (Poisson) /42

9. University of Virginia Department of Systems & Information Engineering 9 Spatially Defined Point Processes For spatially defined PP, events are located in a bounded plane, A   2 –where A is the study region and the events are In our processes, time cannot be ignored –Spatio-temporal model (full model) –Spatial model (time integrated out) –Time series model (space integrated out) Spatial Density (Preferences) Temporal Rate /42

10. University of Virginia Department of Systems & Information Engineering 10 Feature Space The feature space G, defines the additional attribute information relating to the locations in the study region –g(s,t)=[g 1 (s,t),g 2 (s,t),…,g p (s,t)] G defines p other spatially defined stochastic processes –Real (perhaps discrete), ordered, or categorical valued –High Dimensional (large p) The PP is specified conditionally on the values of the other stochastic processes –The intensity is dependent upon the feature values  (s,t)   (s,t|g(s,t)) /42

11. University of Virginia Department of Systems & Information Engineering 11 Feature Space Formulation Since we condition on the value of g it is assumed that given any (s,t), the value g(s,t) can be determined –Using GIS  (s,t)=h(g(s,t)) –g is a realization of G –h is the function we want to determine Two ways to examine model –Define the PP in feature space –Look at association between PP and random processes /42

12. University of Virginia Department of Systems & Information Engineering 12 Overview of the point process model Liu and Brown 1999 /42

13. University of Virginia Department of Systems & Information Engineering 13 Spatial Association / Feature Selection Find the features that are associated with the point process –Correlated with PP, but not necessarily causal –This is preference discovery Two procedures –Find important features –Remove redundant (correlated) features Important Features (Foxall and Baddeley 2002) J(r)=[1-G(r)]/[1-F(r)] =Pr(event>r)/Pr(grid>r) »J® 1 suggests repulsion »Feature is important if J(r) is much different than 1 for small r /42

14. University of Virginia Department of Systems & Information Engineering 14 Values much lower that 1 suggest the PP is associated with feature g 5 Values close to 1 suggest the PP is not associated with feature g 6

15. University of Virginia Department of Systems & Information Engineering 15 Removing Redundant Features If a feature is redundant, its intensity can be explained by another feature Rescaling Theorem –Rescale events {e i }  {e i ’} so e i ’ form homogeneous Poisson with rate  =1 Estimate intensity based on one feature m (perhaps the most important from the last step) Use this to test for homogeneity of rescaled events in feature k (Let r ki =g k (s i ) ) /42 (Diggle, 1985 Applied Statistics)

16. University of Virginia Department of Systems & Information Engineering 16 Redundant Features – con’t Tests for homogeneity or uniform –K function(Cressie 1993) –Nearest Neighbor –KS –Scan Statistics /42

17. University of Virginia Department of Systems & Information Engineering 17 PredictionPrediction Build space-time prediction model  (s,t+1 | g(s,t),  t ) where  t is the history of the PP up to time t Possible assumptions: 1.Feature values stationary / deterministic Implies g(s,t)=g(s) for all t 2.Intensity (or preferences) stationary for next time period Implies  (s,t+1)=  (s,t) or f(s|t+1)=f(s|t) 3.Intensity is space-time separable Implies  (s,t+1)=f(s)  A (t+1) Model spatial as density and temporal as time series /42

18. University of Virginia Department of Systems & Information Engineering 18 Prediction Models Poisson/ regression –Log{  (s)}=  0 +  1 g 1 (s)+  2 g 2 (s)+... –Should use GAM or transformation since distribution of g is non-uniform on A Mixed models 1 –  (s)=  i f i (s;  i ) Spatial Choice Models 2 1 Liu,H. and Brown, D.E. (2004). A New Point Process Transaction Density Model for Space-Time Event Prediction. Systems, Man and Cybernetics, Part C, IEEE Transactions on, Volume 34, Issue 3, Pg.310 - 324 2 Smith, M.A. (2005). CHOICE MODELING OF BOMBING ATTACK SITE SELECTION, Masters Thesis, Dept of Systems and Information Engineering, University of Virginia, May. /42

19. University of Virginia Department of Systems & Information Engineering 19 Change Detection Statistically detect a region, B, of change between two time periods –Detect changes between two spatial point processes Steps 1.Form hypothesis test 2.Select an appropriate test statistic to test hypothesis 3.Specify how region B will be identified for testing 4.Establish significance testing 5.Evaluate the results Method also applicable for detecting differences between two types of events or case-control studies /42

20. University of Virginia Department of Systems & Information Engineering 20 Formulation: Marked Point Process Let the terrorist process be represented by P = {(s i,k i )}, a marked PP on space X = A x K, A  2, K={1,2} Assuming the ground process is a nonhomogeneous Poisson spatial PP and the marks are independent of each other, P can be specified by: An observation is: /42

21. University of Virginia Department of Systems & Information Engineering 21 Hypothesis Test (Assuming equal time periods and no temporal trends) If unequal time periods or temporal trends, set where i is the time period i. Then under the null: * If unequal time periods and no trends,  T ( 1 )/  T ( 2 )=  ( 1 )/  ( 2 ), where  ( i ) is the length of time period i. /42

22. University of Virginia Department of Systems & Information Engineering 22 Hypothesis Test (Assuming equal time periods and no temporal trends) /42

23. University of Virginia Department of Systems & Information Engineering 23 Hypothesis Test (Assuming equal time periods and no temporal trends) /42

24. University of Virginia Department of Systems & Information Engineering 24 Statistical Test – Generalized Likelihood Ratio Assuming equal time periods and no temporal trends { /42

25. University of Virginia Department of Systems & Information Engineering 25 Finding Change Region B defines the geographical region in A where change has occurred in the intensity function of the point process For testing the null hypothesis,  (s)=  Ho, we must identify the region B We do this by searching over A for the region B* that provides the minimum value of T( ,B) Procedure of scanning for extrema regions for significance testing is termed scan process (e.g. scan statistic) /42

26. University of Virginia Department of Systems & Information Engineering 26 Scan Process In general, scan processes are used to detect significant clusters of events while accounting for multiple hypothesis testing Create a window, W x of some geometry and move it over the entire region of interest (  x  X) and calculate some score for each window, S(W x ). The multiple hypothesis problem is resolved by testing on the scan statistic, SS=max x  X S(W x ) –Dependent on geometry selected for W –But multiple geometries can be considered /42

27. University of Virginia Department of Systems & Information Engineering 27 Modified Scan Process for Change Detection Here, we want to find window location and geometry producing the minimum value of the likelihood ratio statistic Thus our scan statistic becomes SS(  ) = min B  B T( ,B) = T( ,B*) where B is the set of windows that we searched over The windows are restricted to be connected sets in map and feature space (i.e. B  X=AxG) –One change region –Hyper-rectangles in the real variables –No restriction on size of window /42

28. University of Virginia Department of Systems & Information Engineering 28 Approximation to scan statistic When the number of events and/or dimensionality of G is large, searching over all possible windows is computationally prohibitive. Instead, find good approximation to scan statistic Search over restricted set B’  B and hope this set includes B*, or a large portion of it Use PRIM to search for B* and thus the value of our scan statistic /42

29. University of Virginia Department of Systems & Information Engineering 29 Patient Rule Induction Method (PRIM) Patient Rule Induction Method PRIM handles high-dimensional and mixed data well Find the boxes (hyper-rectangles) where the response (i.e. T) is low Procedure is to produce a series of boxes, {B k }, decreasing in size, by successively peeling away a subbox in such a manner that each new box B k+1 has the lowest value for T( ,B k+1 ) among all possible subboxes. /42

30. University of Virginia Department of Systems & Information Engineering 30 PRIM – con’t Define the subboxes x j (  k ) is the  k –quantile of the x j values for the data within the current box B k  k  (0,1) dictates the size of the subboxes The optimal subbox for peeling becomes (For catagorical variables) /42

31. University of Virginia Department of Systems & Information Engineering 31 PRIM – con’t This creates a new box B k+1 = B k -b k * Continue peeling until T can no longer be decreased beyond minimum at step k After peeling procedure, pasting step can further improve box –Add subbox in similar manner as peeling To help find global minimum, allow variable  k and run PRIM for J iterations –Produces Leads to test of significance of for change in region /42

32. University of Virginia Department of Systems & Information Engineering 32 Significance – Monte Carlo /42

33. University of Virginia Department of Systems & Information Engineering 33 Simulated Example Examine region A on unit square –Six features identified: G=[g 1,g 2,…,g 6 ] –Features {g 1,g 2,g 3,g 6 } are proximity to nearest landmark –Features {g 4,g 5 } are values of some variable recorded at census tract level Group of insurgents operating in region A –  1 (s)=exp{-c y s 2 -c 1 g 1 (s)+c 5 g 5 (s)}/C 1 –c y=0.5, c 1 =100, c 5 =1,C 1 =15.5 –E[N(A)]=100 Random attacks with intensity  r =15 New group active in 2 –  2 (s)=I(g 2 (s)  0.10,g 3 (s)  0.05, g 4 (s)  0.30)/C 2 –I is indicator function and C 2 =12 –E[N 2 (A)]=35 /42

34. University of Virginia Department of Systems & Information Engineering 34 Simulated Feature Values /42

35. University of Virginia Department of Systems & Information Engineering 35 ProcedureProcedure Thus,  (s,k=1)=  1 (s)+  r for 1 and  (s,k=2)=  1 (s)+  r +  1 (s) for 2 Let  ( 1 )=  ( 2 ) so under null,  (s)=1 everywhere in A Generate 100 realizations of the PP and for each apply PRIM with  k  [0.05,0.15] For each original observation J=500 iterations were performed For each realization of the simulated PP, 99 simulations under H 0 are created by random labeling with f(k=i|s)=0.50 and p-values are calculated Results in set /42

36. University of Virginia Department of Systems & Information Engineering 36 MeasuresMeasures Let E i be the events from insurgency group i, with random events assigned to group 1 (R 2,R 4 ) – region where change occurs (R 1,R 3 ) – events that constitute change /42

37. University of Virginia Department of Systems & Information Engineering 37ResultsResults ROC P D -Prob to Detect Detection declared if /42

38. University of Virginia Department of Systems & Information Engineering 38 IED Attacks in Iraq Major method of attacking U.S. forces in Iraq Responsible for more U.S. deaths than any other attack mode Inexpensive, easy to deploy, and deadly Picture on right shows U.S. troops with IED on March 15, 2004 Picture from http://www.middle-east-online.com/english/?id=9250 /42

39. University of Virginia Department of Systems & Information Engineering 39 IED Change Region N 1 =151 events, N 2 =91 events | 1 |=| 2 | J=500, M=999 Monte Carlo /42

40. University of Virginia Department of Systems & Information Engineering 40 IED Change Region - con’t Region B* contains 48 events from 1 and 0 events from 2 PRIM peeled 16 times on 14 features B* composes approx. 21% of the total area of A Estimated p-value is 0.001 so reject null and conclude change has occurred in region B* /42

41. University of Virginia Department of Systems & Information Engineering 41 ConclusionsConclusions Modeling terrorist processes is difficult yet important Point process approach provides small scale analysis Including terrorist decision making process leads to better models Non-model based change detection method can detect presence of new terrorist group or changing terrorist attack strategies /42

42. University of Virginia Department of Systems & Information Engineering 42 Background: Recent papers modeling criminal or terrorist attacks 1.Y. Xue and D.E. Brown, “Spatial Analysis with Preference Specification for Latent Decision Makers for Criminal Event Prediction,” Decision Support Systems, forthcoming. 2.Porter, M.D. and Brown, D.E., “Detecting Local Regions of Change in High-Dimensional Criminal or Terrorist Point Processes”, forthcoming 3.Porter, M.D. and Brown, D.E., “Finding Changing Crime Regions: Use of High Dimensional Geographic Feature Space and Classification Trees”. Proceedings of the Eight Crime Mapping Research Conference, Sept. 7-10, 2005 Savannah, GA. 4.Smith, M.A., “Choice Modeling of Bombing Attack Site Selection,” Masters Thesis, Dept of Sys. and Info Engineering, University of Virginia, May 2005. 5.H. Liu and D.E. Brown, “A New Point Process Transition Density Model for Space-Time Event Prediction,” IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews,Vol.34, No.3, 2004, pp. 310-324. 6.D.E. Brown, J. Dalton, and H. Hoyle, “Spatial Forecast Methods for Terrorist Events in Urban Environments,” Second NSF/NIJ Symposium on Intelligence and Security Informatics, Tucson, AZ, June 2004, pp. 426-435. 7.Liu, H. and D.E. Brown, “Criminal Incident Prediction Using a Point-pattern Based Density Model” Int. Journal of Forecasting, vol. 19 (4), 2003, pp. 603-622. 8.Xue, Y. and D.E. Brown, “A Decision Model for Spatial Site Selection by Criminals: A Foundation for Law Enforcement Decision Support,” IEEE Transactions on System, Man, and Cybernetics, Part C: Applications and Reviews, Vol. 33, No. 1, February 2003, pp. 78-85. /42