1. 1 Biosurveillance Sensor Networks and Resultant Spatiotemporal Data for Crisis-Index Development and Early Warning Austin, March 2005 G. P. Patil Austin, March 2005

2. 2 Infectious Diseases Use of remote sensing data and other available geospatial data on a continental scale to help evaluate landscape characteristics that may be precursors for vector-borne diseases leading to early warning systems involving landscape health, ecosystem health, and human health Use of remote sensing data and other available geospatial data on a continental scale to help evaluate landscape characteristics that may be precursors for vector-borne diseases leading to early warning systems involving landscape health, ecosystem health, and human health Water-Borne Diseases Water-Borne Diseases Air-Borne Diseases Air-Borne Diseases Emerging Infectious Diseases Emerging Infectious Diseases

3. 3

4. 4 Dead Bird Clusters as an Early Warning System for West Nile Virus Activity Mostashari, Kulldorff, Hartman, Miller, Kulasekara, 2003. Emerging Infectious Diseases 9, 641-646. Prospective geographic cluster analysis of dead bird reports may provide early warning of increasing viral activity in birds and mosquitoes, allowing jurisdictions to triage limited mosquito-collection and laboratory resources and more effectively prevent human disease caused by the virus… We adapted the spatial scan statistic for prospective detection of infectious disease outbreaks in New York City… This adaptation of the spatial scan statistic could also be useful in other infectious disease surveillance systems, including those for bioterrorism. Prospective geographic cluster analysis of dead bird reports may provide early warning of increasing viral activity in birds and mosquitoes, allowing jurisdictions to triage limited mosquito-collection and laboratory resources and more effectively prevent human disease caused by the virus… We adapted the spatial scan statistic for prospective detection of infectious disease outbreaks in New York City… This adaptation of the spatial scan statistic could also be useful in other infectious disease surveillance systems, including those for bioterrorism.

5. 5 Geographic Prediction of Human Onset of West Nile Virus Using Dead Crow Clusters: A Quantitative Assessment of Year 2002 Data in New York State Johnson, Eidson, Schmit, Ellis, Kulldorff (2005) The risk of becoming a West Nile Virus (WNV) case in New York State, excluding New York City, was evaluated for individuals whose town of residence was proximal to spatial clusters of dead American crows (Corvus brachyrhynchos). Weekly clusters were delineated for June-October 2002 by the binomial spatial scan statistic and by kernel density smoothing. Relative risk of a human case was estimated for different spatial-temporal exposure definitions after adjusting for population density and age distribution using Poisson regression, adjusting for week and geographic region, and Cox proportional hazards modeling, where the week of a human case was treated as the failure time and baseline hazard was stratified by region.

6. 6 The risk of becoming a WNV case was positively associated with living in towns that are proximal to dead crow clusters. The highest risk was consistently for towns associated with a cluster in the current or prior 1-2 weeks Weaker, but positive, associations were found for towns associated with a cluster in just the 1-2 prior weeks, indicating an ability to predict onset in a timely fashion. These results are relevant to spatial-temporal surveillance for zoonotic diseases in general, including those of bioterrorism concern.

7. 7 Geospatial Patterns and Pattern Metrics Geospatial Patterns and Pattern Metrics –Landscape patterns, disease patterns, mortality patterns Surface Topology and Spatial Structure Surface Topology and Spatial Structure –Hotspots, outbreaks, critical areas –Intrinsic hierarchical decomposition, study areas, reference areas –Change detection, change analysis, spatial structure of change Multiscale Advanced Raster Map Analysis

8. 8 MARMAP SYSTEM Partnership Research and Outreach Prospectus: http://www.stat.psu.edu/~gpp/PDFfiles/pro spectus8-00.pdf Partnership Research and Outreach Prospectus: http://www.stat.psu.edu/~gpp/PDFfiles/pro spectus8-00.pdf http://www.stat.psu.edu/~gpp/PDFfiles/pro spectus8-00.pdf http://www.stat.psu.edu/~gpp/PDFfiles/pro spectus8-00.pdf Our web page for raster map analysis: http://www.stat.psu.edu/~gpp/newpage11. htm Our web page for raster map analysis: http://www.stat.psu.edu/~gpp/newpage11. htm http://www.stat.psu.edu/~gpp/newpage11. htm http://www.stat.psu.edu/~gpp/newpage11. htm Our web page for raster map monographs: http://www.stat.psu.edu/~gpp/raster.htm Our web page for raster map monographs: http://www.stat.psu.edu/~gpp/raster.htm http://www.stat.psu.edu/~gpp/raster.htm Our web page for UNEP HEI Our web page for UNEP HEI http://www.stat.psu.edu/~gpp/unephei.htm http://www.stat.psu.edu/~gpp/unephei.htm http://www.stat.psu.edu/~gpp/unephei.htm

9. 9

10. 10 National Applications Biosurveillance Carbon Management Coastal Management Community Infrastructure Crop Surveillance Disaster Management Disease Surveillance Ecosystem Health Environmental Justice Environmental Management Environmental Policy Homeland Security Invasive Species Poverty Policy Public Health Public Health and Environment Robotic Networks Sensor Networks Social Networks Syndromic Surveillance Tsunami Inundation Urban Crime Water Management

11. 11 Examples of Hotspot Analysis Spatial Disease surveillance Biodiversity: species-rich and species-poor areas Geographical poverty analysis Network Water resource impairment at watershed scales Water distribution systems, subway systems, and road transport systems Social Networks/Terror Networks

12. 12 Issues in Hotspot Analysis Estimation: Identification of areas having unusually high (or low) response Testing: Can the elevated response be attributed to chance variation (false positive) or is it statistically significant? Explanation: Assess explanatory factors that may account for the elevated response

13. 13 The Spatial Scan Statistic Move a circular window across the map. Move a circular window across the map. Use a variable circle radius, from zero up Use a variable circle radius, from zero up to a maximum where 50 percent of the population is included.

14. 14 A small sample of the circles used

15. 15 Detecting Emerging Clusters Instead of a circular window in two dimensions, we use a cylindrical window in three dimensions. Instead of a circular window in two dimensions, we use a cylindrical window in three dimensions. The base of the cylinder represents space, while the height represents time. The base of the cylinder represents space, while the height represents time. The cylinder is flexible in its circular base and starting date, but we only consider those cylinders that reach all the way to the end of the study period. Hence, we are only considering ‘alive’ clusters. The cylinder is flexible in its circular base and starting date, but we only consider those cylinders that reach all the way to the end of the study period. Hence, we are only considering ‘alive’ clusters.

16. 16 Hospital Emergency Admissions in New York City Hospital emergency admissions data from a majority of New York City hospitals. Hospital emergency admissions data from a majority of New York City hospitals. At midnight, hospitals report last 24 hour of At midnight, hospitals report last 24 hour of data to New York City Department of Health A spatial scan statistic analysis is performed every morning A spatial scan statistic analysis is performed every morning If an alarm, a local investigation is conducted If an alarm, a local investigation is conducted

17. 17 Syndromic Surveillance Symptoms of disease such as diarrhea, respiratory problems, headache, etc Symptoms of disease such as diarrhea, respiratory problems, headache, etc Earlier reporting than diagnosed disease Earlier reporting than diagnosed disease Less specific, more noise Less specific, more noise New York City New York City Pennsylvania Pennsylvania

18. 18 (left) The overall procedure, leading from admissions records to the crisis index for a hospital. The hotspot detection algorithm is then applied to the crisis index values defined over the hospital network. (right) The -machine procedure for converting an event stream into a parse tree and finally into a probabilistic finite state automaton (PFSA). Syndromic Surveillance

19. 19 Crisis-Index Surveillance

20. 20         ……  …… Phase-Space TrajectoryString of Symbols Symbolization: Network Sensor Readings to Symbolic Dynamics Sensor 1 Sensor 2 Sensor 3

21. 21 Experimental Validation Pressure sensitive floor Formal Language Events: a – green to red or red to green b – green to tan or tan to green c – green to blue or blue to green d – red to tan or tan to red e – blue to red or red to blue f – blue to tan or tan to blue Wall following Random walk Analyze String Rejections Target Behavior

22. 22 Emergent Surveillance Plexus (ESP) Surveillance Sensor Network Testbed Autonomous Ocean Sampling Network Types of Hotspots Hotspots due to multiple, localized, stationary sources Hotspots due to multiple, localized, stationary sources Hotspots corresponding to areas of interest in a stationary mapped field Hotspots corresponding to areas of interest in a stationary mapped field Time-dependent, localized hotspots Time-dependent, localized hotspots Hotspots due to moving point sources Hotspots due to moving point sources

23. 23 Ocean SAmpling MObile Network OSAMON

24. 24 Ocean SAmpling MObile Network OSAMON Feedback Loop Network sensors gather preliminary data Network sensors gather preliminary data ULS scan statistic uses available data to estimate hotspot ULS scan statistic uses available data to estimate hotspot Network controller directs sensor vehicles to new locations Network controller directs sensor vehicles to new locations Updated data is fed into ULS scan statistic system Updated data is fed into ULS scan statistic system

25. 25 SAmpling MObile Networks (SAMON) Additional Application Contexts Hotspots for radioactivity and chemical or biological agents to prevent or mitigate the effects of terrorist attacks or to detect nuclear testing Hotspots for radioactivity and chemical or biological agents to prevent or mitigate the effects of terrorist attacks or to detect nuclear testing Mapping elevation, wind, bathymetry, or ocean currents to better understand and protect the environment Mapping elevation, wind, bathymetry, or ocean currents to better understand and protect the environment Detecting emerging failures in a complex networked system like the electric grid, internet, cell phone systems Detecting emerging failures in a complex networked system like the electric grid, internet, cell phone systems Mapping the gravitational field to find underground chambers or tunnels for rescue or combat missions Mapping the gravitational field to find underground chambers or tunnels for rescue or combat missions

26. 26 Scalable Wireless Geo-Telemetry with Miniature Smart Sensors Geo-telemetry enabled sensor nodes deployed by a UAV into a wireless ad hoc mesh network: Transmitting data and coordinates to TASS and GIS support systems

27. 27 Architectural Block Diagram of Geo-Telemetry Enabled Sensor Node with Mesh Network Capability

28. 28 Data Fusion Hierarchy for Smart Sensor Network with Scalable Wireless Geo-Telemetry Capability

29. 29 Target Tracking in Distributed Sensor Networks

30. 30 Wireless Sensor Networks for Habitat Monitoring

31. 31 Video Surveillance and Data Streams

32. 32 Video Surveillance and Data Streams Turning Video into Information Measuring Behavior by Segments Customer Intelligence Enterprise Intelligence Entrance Intelligence Media Intelligence Video Mining Service

33. 33 Key Crop Areas Crops NOAA Weather Threat Locations Plants Infected Non-infected Sentinel Ground Cameras Air/Space Platforms Hyperspectral Imagery Signature Library Data Processing Anomaly Report Crop Attack Decision Support System Ground Truthing Site Identification Module Signature Development Module

34. 34 Crop Biosurveillance/Biosecurity

35. 35 Hyperspectral Imagery Signature Library Image Segmentation (hyperclustering) Proxy Signal (per segment) Disease Signature Similarity Index (per segment) Tessellation (segmentation) of raster grid Signature Similarity Map Hotspot/ Anomaly Detection Crop Biosurveillance/Biosecurity Data Processing Module

36. 36 Space-Time Poverty Hotspot Typology Federal Anti-Poverty Programs have had little success in eradicating pockets of persistent poverty Federal Anti-Poverty Programs have had little success in eradicating pockets of persistent poverty Can spatial-temporal patterns of poverty hotspots provide clues to the causes of poverty and lead to improved location-specific anti- poverty policy ? Can spatial-temporal patterns of poverty hotspots provide clues to the causes of poverty and lead to improved location-specific anti- poverty policy ?

37. 37 Dimensions of Tract Poverty in Four Metropolitan Areas 1970-1990 Concentrated Persistent Camden, NJ Detroit, MI Oakland, CA Memphis, TN Growing Shifting

38. 38 Growing poverty

39. 39 Persistent poverty

40. 40 Oakland 1970 Poverty dataOakland 1980 Poverty data Oakland 1990 Poverty data Shifting poverty

41. 41 Camden 1990 Poverty data Concentrated poverty Camden 1970 Poverty dataCamden 1980 Poverty data

42. 42 Hotspot Persistence Hotspot Persistence Space (census tract) 1970 1980 1990 2000 Time (census year) Persistent Hotspot Long Duration Space (census tract) 1970 1980 1990 2000 Time (census year) One-shot Hotspot Short Duration Persistence is a property of space-time hotspots Persistence can be assessed by the projection of the space-time hotspot onto the time axis

43. 43 Typology of Persistent Space-Time Hotspots-1 Space (census tract) 1970 1980 1990 2000 Time (census year) Stationary Hotspot Space (census tract) 1970 1980 1990 2000 Time (census year) Shifting Hotspot Space (census tract) 1970 1980 1990 2000 Time (census year) Expanding Hotspot Space (census tract) 1970 1980 1990 2000 Time (census year) Contracting Hotspot

44. 44 Typology of Persistent Space-Time Hotspots-2 Space (census tract) 1970 1980 1990 2000 Time (census year) Bifurcating Hotspot These hotspots are connected in space-time However, certain time slices of the hotspot are disconnected in space Space (census tract) 1970 1980 1990 2000 Time (census year) Merging Hotspot Spatially disconnected time slice

45. 45 Trajectory of a Persistent Space-Time Hotspot A space-time hotspot is a three-dimensional object Visualization can be done by displaying the sequence of time slices---called the trajectory of the hotspot Time slices of space-time hotspot Space (census tract) 1970 1980 1990 2000 Time (census year) Merging Hotspot

46. 46 Trajectory of a Merging Hotspot 19701980 1990 2000

47. 47 Trajectory of a Shifting Hotspot 19701980 1990 2000

48. 48 Tsunami Inundation and Evacuation

49. Penn State Cooperative Wetlands Center 49 Application of the Scan Statistic to Wetland and Stream Monitoring: Testing the Method in the Upper Juniata Watershed, Pennsylvania, USA Denice Heller Wardrop, Charles Taillie, Kristen Hychka, G.P. Patil, and Wayne Myers Penn State University, University Park, PA

50. Penn State Cooperative Wetlands Center 50 Potential advantages of critical areas as an indicator of stress Delineation of contributing areas is labor intensive Delineation of contributing areas is labor intensive Can critical areas give us a first cut as to wetlands in poor condition? Can critical areas give us a first cut as to wetlands in poor condition? Can they serve as an early warning? Can they serve as an early warning?

51. Penn State Cooperative Wetlands Center 51

52. Penn State Cooperative Wetlands Center 52 Restoration Targeting with SaTScan Can effectively target areas for restoration activities or best management practices Can effectively target areas for restoration activities or best management practices Can be scaled so that critical area matches management area Can be scaled so that critical area matches management area

53. Penn State Cooperative Wetlands Center 53 Network Analysis of Biological Integrity in Freshwater Streams

54. Penn State Cooperative Wetlands Center 54 Network-Based Surveillance Subway system surveillance Subway system surveillance Drinking water distribution system surveillance Drinking water distribution system surveillance Stream and river system surveillance Stream and river system surveillance Postal System Surveillance Postal System Surveillance Road transport surveillance Road transport surveillance Syndromic Surveillance Syndromic Surveillance

55. Penn State Cooperative Wetlands Center 55

56. 56

57. 57

58. 58 Covariate Adjustment Known Covariate Effects (age, population size, etc.)

59. 59 Covariate Adjustment Given Covariates, Unknown Effects

60. 60 Incorporating Spatial Autocorrelation Ignoring autocorrelation typically results in:  under-assessment of variability  over-assessment of significance (H 0 rejected too frequently) How can we account for possible autocorrelation? GLMM (SAR) Model Y a = count in cell a Y a distributed as Poisson  a = log(E[Y a ]) The Y a are conditionally independent given the  a The  a are jointly Gaussian with a Simultaneous AutoRegressive (SAR) specification

61. 61 Incorporating Spatial Autocorrelation

62. 62 Incorporating Spatial Autocorrelation

63. 63 Spatial Autocorrelation Plus Covariates

64. 64 CAR Model The entire formulation is similar for Conditional AutoRegressive (CAR) specs except that the form of the variance-covariance matrix of  is changes.

65. 65

66. 66 Deterministic Finite Automata (DFA) a a b b b c c start Directed Graph (loops & multiple edges permitted) such that: Nodes are called States Edges are called Transitions Distinguished initial (or starting) state Transitions are labeled by symbols from a given finite alphabet,  = {a, b, c,... } The same symbol can label several transitions A given symbol can label at most one transition from a given state (deterministic)

67. 67 Deterministic Finite Automata (DFA) Formal Definition a a b b b c c start Quadruple (Q, q 0, ,  ) such that: Q is a finite set of states  is a finite set of symbols, called the alphabet q 0  Q is the initial state  : Q    Q  {Blocked} is the transition function:   (q, a) = Blocked if there is no transition from q labeled by a   (q, a) = q' if a is a transition from q to q'

68. 68 DFA and Strings a a b b b c c start Any path through the graph starting from the initial state determines a string from the alphabet. Example: The blue dashed path determines the string a b c a Conversely, any string from the alphabet is either blocked or determines a path through the graph. Example: The following strings are blocked: c, aa, ac, abb, etc. Example: The following strings are not blocked: a, b, ab, bb, etc. The collection of all unblocked strings is called the language accepted or determined by the DFA (all states are “final” in our approach)

69. 69 Strings and Languages  = (finite) alphabet  * = set of all (finite) strings from  A language is any subset of  *. Not all languages can be determined by a DFA. Different DFAs can accept the same language

70. 70 Probabilistic Finite Automata (PFA) A PFA is a DFA (Q, q 0, ,  ) with a probability attached to each transition such that the sum of the probabilities across all transitions from a given node is unity. Formally, p: Q    [0, 1] such that p(q, a) = 0 if and only if  (q, a) = Blocked Multiplying branch probabilities lets us assign a probability value  (q 0, s) to each string s in  *. E.G.,  (q 0, abca)=(.8)1(.6)(.4)=.192 q0q0 a,.4 b,.2 b, 1 b,.5 c,.6 c,.5 start a,.8

71. 71 Properties of  (q 0, s) For fixed q 0,  (q 0, s) is a measure on  * Support of  is the language accepted by the DFA For fixed q 0,  (q 0, s) is a probability measure on  i (  i = strings of length i ) This probability measure is written as  (i). Given a probability distribution w(i) across string lengths i, defines a probability measure across  *, called the w-weighted probability measure of the PFA. If all w(i) are positive, then the support of  is also the language accepted by the underlying DFA.

72. 72 Distance Between Two PFA Let A and B be two PFAs on the same alphabet  Let w(i) be a probability distribution across string lengths i Let  A and  B be the w-weighted probability measures of A and B Define the distance between A and B as the variational distance between the probability measures  A and  B : d( A, B) = ||  A   B ||

73. 73 Using  -complexity for Network Behavior Analysis David Friedlander (dsf10@psu.edu) Shashi Phoha (spx26@psu.edu) Richard Brooks (rrb5@psu.edu) Penn State / ARL

74. 74 Tools for Recognizing Target Behavior From Network Measurements Symbolization Conversion Behavior Recognition Network Measurements (streams of numbers) Stream of symbols Higher level representation Representations of Known Behaviors Target Behavior

75. 75 Natural Language Definition (Merriam- Webster’s Collegiate  Dictionary) Behavior: 1b : anything that an organism does involving action and response to stimulation c : the response of an individual, group, or species to its Technical Definitions Behavior → Pattern of observations and actions Pattern → Formal language Observations → Uncontrollable events Actions → Controllable events

76. 76         ……  …… Phase-Space TrajectoryString of Symbols Symbolization: Network Sensor Readings to Symbolic Dynamics Sensor 1 Sensor 2 Sensor 3

77. 77 ……  …… Conversion Tools: Stream of Symbols to FSA Which defines a formal language of the target behavior

78. 78 a a,b ab 3.Merge topologically similar subtrees ……abaaabaaababababaaabab…. Conversion via topological complexity method 1. Language Sample 2. Tree of all substrings of length l.

79. 79 Conversion via  -complexity method a, P(a|0) b, P(b|0) a aa 0 12 3 4 5 67 8 9 b, P(b|2) a, P(a|2) b, P(b|3)a, P(a|3)  -complexity ……abaaabaaababababaaabab…. 1.Language Sample 2.Tree of all substrings of length l with transition probabilities 3.Merged subtrees must be topologically similar and have similar probability structures

80. 80 Behavior Classification Tool using Finite State Automata ……  …… Behavior 1 Behavior 2 String Rejections (1) / Sec String Rejections (2) / Sec Analyze Rejection Rates to find most likely known behavior for the sample (if any are close enough) Sample taken over “short” time scale Dynamic Target Behavior Changes over “long” time scale

81. 81 Conversion Tools: Formal Languages to Infinite Dimensional Vector Space For example: …..abababaaababaaaba….. Measures are defined on the vector space that satisfy: The space contains a vector for all possible languages of a given alphabet:

82. 82 Weighted Counting Measure for Formal Languages Various measures can be defined on the formal language vectors, such as: where n i (L) is the number of strings of length i in language L, and where k is the number of symbols in the alphabet. The distance between two languages is defined as:

83. 83 Behavior Classification Tool using a Formal Language Measure ……  …… Sample taken over “short” time scale Dynamic Target Behavior Changes over “long” time scale Convert to Vector Vectors of known behaviors

84. 84 Future Work – Recognizing the Behavior of Multiple Targets Recognizing the behaviors of multiple targets Stages Finding stationary targets Finding moving targets Recognizing behaviors of multiple targets Methods Sensor energy surface Sensor cross-correlation

85. 85 Future Work – Recognizing multiple targets – Method 1 Sensor energy surface

86. 86 Future Work – Recognizing multiple targets – Method 2 Sensor cross-correlation

87. 87 Future Work – Behavior Recognition of Multiple, Coordinated Enemy Assets Can we the extend model recognition techniques to hierarchical control systems?