Presentation is loading. Please wait.

Presentation is loading. Please wait.

Situation Understanding & Threat Prediction R. Antony SAIC VISE Support to Kinematics + ID = Understanding KinematicsClassificationIDObjective/IntentThreatBehavior.

Similar presentations

Presentation on theme: "Situation Understanding & Threat Prediction R. Antony SAIC VISE Support to Kinematics + ID = Understanding KinematicsClassificationIDObjective/IntentThreatBehavior."— Presentation transcript:

1 Situation Understanding & Threat Prediction R. Antony SAIC VISE Support to Kinematics + ID = Understanding KinematicsClassificationIDObjective/IntentThreatBehavior Organization

2 Agenda Motivational examples Vision-Inspired Spatial Engine (VISE) Emulates key characteristics of human-level reasoning Crisp spatial reasoning Fuzzy spatial reasoning Mostly theoretical work to date Important applications to robust decision making Comprehensive examples Understanding behavior Demo (data set conflation/registration) Data mining (notional IED predict/detect) Demo (fuzzy-based reasoning) Global planning (optional) Top-down reasoning Mixed spatial/non-spatial reasoning Summary

3 Commander’s Guidance AND no foliage”“Within 1 km of roads AND low slope

4 “Within 1 km of roads AND low slope AND no foliage Concealment Potential for Focusing Collection and Cueing Applying relevant constraints can focus the analysis process, improving both algorithm efficiency & robustness AND concealment” Only regions that meet all criteria

5 No location discrimination Avoid water No-go Avoid heavy foliage, slow-go, or no-go 1 km from roads, low slope, no foliage Must search all white area Constrained Search Region

6 Low accuracy track Kinematics only No ID Context Assessment Interpreting/Refining Behavior, Intent Estimation Mid-level context Predict road-seeking Predict river- crossing objective Global context Seek Checkpoint Bravo Assume minimum gradient travel Predict road- following Local context Possible hide Possible concealment intent Predict road- following

7 Low accuracy track Kinematics only No ID Context Assessment Behavior/Intent Understanding Road-seeking River- crossing Seek Village Bravo Minimum gradient travel Road- following Hide Concealment Road- following Unknown vehicle Possible hostile Unknown vehicle Possible hostile Tracked or wheeled Likely wheeled vehicle High-speed, suspicious wheeled vehicle approaching checkpoint

8 Refining Intent Estimation (Vehicle) Added Constraints Concealment No-go Slow-go Concealment Avoid UGS field Concealment New vehicle track demonstrates increased hostile intent

9 Refining Intent Estimation (Dismount) Context-Sensitive Interpretation Concealment No-go Slow-go Dismount demonstrates deliberate concealment - possible hostile intent Concealment Avoid UGS field Hide MIN terrain gradient Cross at bridges

10 Sensor-derived information is only a part of the Intel “equation” True understanding demands effective reasoning & appropriate domain knowledge (in the form of rules, databases, constraints, models - from concrete to the ephemeral) Application of complex constraints (semantic, temporal & spatial, fuzzy & crisp) Assessment of possibility, plausibility & probability Interpretation of perceived behavior with respect to expectations Data mining to deduce non-obvious correlations/patterns Search-oriented support to knowledge discovery Observations Kinematics + ID = Understanding KinematicsClassificationIDObjective/IntentThreatBehavior Organization Sensor data richKnowledge rich

11 Conventional Machine-Based Reasoning Shortcomings Robustness Brittle, unreliable, overly simplistic approaches Data & knowledge poor Efficiency Computational requirements Combinatorial hypothesis explosion Timeliness of results Context sensitivity Under-constrained solutions Illogical / inappropriate conclusions Related issues Reliability Accuracy/precision Adaptability Flexibility Metrics

12 Vision-Inspired Spatial Engine (VISE) A human analyst-motivated reasoning paradigm Intuitive reasoning across mixed data types Spatial - vector & raster Semantic Effective spatial “window” into a database Efficient evaluation of all spatial relations Absolute (distance, direction, enclosed, outside) Fuzzy (near, large, high) Top-down, highly focused reasoning Global/multiple level of abstraction “Multi-resolution” reasoning with vector data Multiple level of abstraction hypothesis management Metrics sensitive to data resolution (raster) / precision (vector) Supports Fusion/conflation Data mining Perceptual-based spatial similarity match

13 VISE Engine Spatial Similarity match Vector set operation Top-down reasoning Video exploitation GIS/Image conflation GIS/GIS conflation Constraint -based reasoning Image mosaic Motion tracking Motion extraction Database indexing VISE Program Efficient spatial reasoning & hypothesis management Applications

14 US Army TEC Disparate resolution road conflation US Army CECOM Design & code set operation algorithm US Army CECOM Video MTI NRO Image registration NRO/CIA Geospatial-Image linking/registration US Army Research Lab (consulting & support to Warrior’s Edge) DARPA (SETA support DDB/DTT) VISE Program Timeline Pre-History Fusion research: Department of the Army ( ) Principles of Data Fusion Automation, Artech House, 1995 Research findings let directly to VISE development 2005

15 Commentary Effective context utilization Helps avoid point solutions Corrects/improves under-constrained solutions Typically non-sensor derived information Intrinsic problem dimensions must match solution dimensions Match technology to problem (numerical, graph search, rule-based, procedural, …) Emulate (macro-level) human-style reasoning when other methods prove inadequate Database management May involve large spatially & non-spatially organized data sets Database organization can (positively or negatively) impact algorithm development

16 Reality v. Representation Extracted image feature Raster-based image (resolution dependent) Real features are inherently areal (lines and points are mathematical abstractions - degenerate regions) Humans are adept at multi-resolution reasoning with both areal and map-based representations Vector representations offer both high precision and memory efficiency Vector-based geospatial feature Mathematically, an infinitely thin line Map-based depiction of feature Line color and thickness keyed to feature class

17 Polygon Representations What the computer “sees” (vector tuple representation) (x 1, y 1 ), (x 2, y 2 ), (x 3, y 3 ), (x 4, y 4 ), … VISE uses a unique spatial indexing structure that permits algorithms to “see” edges and area at multiple resolution levels Spatially-organized depiction of these points Piecewise linear shape What a human sees Edges & internal area

18 Spatial Data Organization Efficient Access & Manipulation Region Quadtree Multi-resolution decomposition of 2-D space Inherently preserves spatial relationships

19 Effectively integrates raster and vector representations 1. Point 2. Line 3. Region boundary cell 4. Region interior cell Quadtree-Indexed Vector Representation Underlying VISE Spatial Representation

20 Successive Refinement Point feature Quadtree-Indexed Vector Spatial Representation Indexing cells provide multi-resolution representation of tuple

21 Example #1 Data Mining/Understanding Behavior/Prediction Sensor-derived information Understanding Stop location is near building 543 Spatial-only association Source 10 minute confirmed stop Target XYZ track & kinematics Radar Large moving vehicle with max velocity V m Vehicle has 12 cylinders UGS One of five possible vehicle classes Image chip (EO) Tank class X Target may have been in hide Predominately off-road travel (supports tracked vehicle hypothesis) Possible route concealment intent Derived context 10-minute stop occurred in vicinity of existing “stopped” target Probable rendezvous Possible meeting took place Spatial/temporal correlation

22 Road intersection UGS site Choke point Too steep Marsh Understanding the Domain Hide Derivable context Hide Off-road travel Follow roads Avoid roads near intersection

23 Low Slope Degrees 100% 80% Using Domain Knowledge Fuzzy Membership Functions

24 Target detected moving from hide Possible goal state Minimum distance path Global Path Planning/Plan Recognition Domain-Sensitive Reasoning

25 Global Path Planning/Plan Recognition Assessing Coarse-Level Domain Constraints Target detected moving from hide Possible goal state VISE supports efficient identification of barriers to ground mobility

26 Road intersection UGS site Choke point Too steep Marsh Domain-Sensitive Reasoning Local Analysis Hide 100% Near definition 80% 60%

27 Road intersection UGS site Choke point Too steep Marsh Identifying Local Domain Knowledge Hide Local: slope low; ground firm, good off-road mobility Less local: low to moderate slope, firm & rocky Moderate: low to moderate slope, firm, rocky & marsh

28 Road intersection UGS site Too steep Marsh Hide Local: Road, low slope, river Less local: Road, low slope, river, marsh Moderate: Road, low slope, firm soil, river, marsh, bridge Identifying Local Domain Knowledge Choke point

29 Trafficability Considerations Color-Coded by Elevation High Moderate Low Very low

30 Understanding Behavior DEM Coded by Elevation High Moderate Low Very low Indicates low gradient path Low slope Moderate Steep

31 Trafficability Considerations Track not Consistent with Ground Mover

32 Understanding Behavior DEM Coded by Elevation High Moderate Low Very low Mixed gradient path - low probability of being a ground-based vehicle High probability of hostile if low speed during high gradient segments Low slope Moderate Steep

33 Spatial/Temporal Context Data Mining Stop location Discover Building % Near definition 80%

34 VISE Support Mixed Spatial & Semantic Reasoning River 1 Spatial Object Database Semantic Object Database Rivers Roads Road 1 Road Quadtree-indexed vector representation DrainageBuildings Hand-off from spatial to semantic reasoning

35 Target ABC is currently stopped at location DEF Relevant associations Understanding Behavior Mining Context Sensor-derived information Understanding Source Building 543 is suspected insurgent warehouse HUMINT/Intel Possible weapon pickup or refueling Semantic & spatial data mining Building 543 is near heavily forested area Building is 12,000 square feet, single story, built in 1982 Registered business is farm equipment repair Feasible cover/staging area near-by Two major roads are within 0.5km of site Off-road travel in vicinity difficult for wheeled vehicles Prior arrest on drug charges of two individuals working in management positions Historical perspective Track of XYZ nearly identical to Track of ABC

36 Prediction/extrapolation Likely Target XYZ objective: location DEF Understanding Behavior Mining Context Sensor-derived information Understanding Source Location DEF is choke point Location DEF is a possible Blue convoy ambush location Hypothesis consistent with statistically- based predictive knowledge Threat assessment/ Threat analysis Note the lack of sensor-derived information driving the latter stages of this analysis

37 Discover bridge Infer possible choke point Local Context Evaluation Data Mining

38 Vehicle under track stops (radar) P erson emerges (radar/IR confirmation/EO ID) Individual walks (to building) Dismount disappears (into building) After 30 minutes, dismount [same or different (EO)] (leaves building) Dismount walks to a location and (gets into the same vehicle) Vehicle departs Without context there would be no justification for connecting track fragments separated by 30 minutes There would be no way to infer that a meeting or transaction might have occurred In short, there would be little understanding Kinematics + ID = Understanding Understanding Exploitation of “Embedded” Context

39 Inbound vehicle track Inbound human Outbound human “Understanding” Sensor Data Knowledge of building helps explain: Reason for vehicle track (destination) Outbound vehicle track Reason for vehicle stopping (allow occupants to enter building) Reason for extended dismount disappearance Meeting or exchange may have taken place Context allows fragmentary information to be stitched together to tell a meaningful story

40 VISE Conflation/Registration Capability Supports image-to-image, vector-to-image, and vector-to-vector conflation Relies on quadtree-indexed vector set operation engine Linked vector-based geospatial features from disparate databases (US Army TEC) Linked thousands of complex vector-represented features to overhead imagery (NRO) Performs perceptual-based spatial similarity match by simultaneously matching both edge and area Works for arbitrary-shaped features Can detect missing & changed features Highly intuitive evaluation metrics are sensitive to data resolution/precision Previous example assumed properly aligned data layers If data layers aren’t adequately registered, poor results can be expected

41 Green: PITD (1:250K) Blue: ITD (1:50K) Feature Conflation Link features between and across data layers Application: Find blue features that associate with selected green feature

42 Green: PITD (1:250K) Blue: ITD (1:50K) Link Criteria Feature class Distance Route continuity Shape similarity Link Solution Work performed for US Army TEC Associated ITD Feature

43 VISE Demo #1 VISE Spatial Similarity Match IGR ->HiRes MOUT Searching an image for vehicles near a particular building requires that image and GIS building layers be precisely registered VISE conflation tool can link individual features with their image counterparts Disparate precision data Simultaneous area & edge match

44 Near conf = 0.8 Register Individual Image Feature with GIS Ineffective if near window is applied to unregistered image Application: Find all vehicles near Building A5 Building A5 Building-level registration Near conf = 1.0

45 Flexible Level 1-4 Fusion Toolkit Data mining, sensor control, prediction, situation understanding Query support Query on content/context Max, min, type, feature value, range, statistical attributes Query on abstract features (abstract-to-specific, local-to-global) Barriers, routes, hide, mobility, concealment Query with respect to arbitrary spatial relations (fuzzy and absolute) Spatial windowing Set operations (concatenate constraints) Arbitrary combinations of vector point, line, region features & imagery Effective situation generalization supports understanding of behavior & intent Build and apply complex rule sets & filters Flexible rule-based constraint concatenation Alerts, triggers, search conditions Absolute & relative constraints Key characteristics Allow dynamically adjustable, operator-specified constraints/evaluation criteria Support global-to-local analysis Accommodate mixed spatial & non-spatial attributes Visualize impact of multiple problem dimensions/conditions High execution efficiency facilitates effective “what if” analysis

46 Probability of future attack overlays (based on historical evidence) Relations to convoy route Shopping districts Vehicle build-up near compound Gradient impact on convoy High/low priority search regions Blue Force garrison, supply lines “Approaching” Low level context - soil, gradient, road surface, road class Mid level context- bridge, building, road High level context “Receding/leaving/retreating” Demographics City services Infrastructure Sunni, Shiite owned commercial/residential Topography Recent hotspots Economic districts/other factors Religious districts/mosques/ Schools Local government facilities (Police stations) … Features of Possible Interest

47 COMINT ELINT Buildings Slope forcing vehicles to remain on road “Broken down” vehicle (IMINT) Buildings where insurgents would most likely have spotters IED Example Data Mining, Understanding Behavior, Prediction IEDs Moderate vegetation for access, hide & escape Mosque Within moderate to high risk zone Convoy Potentially wide range of other spatial/non-spatial constraints

48 IED Example - Key Domain Features Low slope High slope Representation of this domain knowledge Low slope High slope Road Buildings Mosque Forest Car

49 VISE-based set operations Generalization of individual features (multi-resolution representation of vector features) Boolean set operation generation Fuzzy set functions (near, almost, small, …) Fuzzy set operations Demo IGR->SO (ARL demo) IGR->SO (Briefing) VISE Demo #2

50 Low-level features Roads Slope Soil type Forest Houses Businesses Utilities Government facilities Abstracted features Mobility (roads, trails, overland) Hide (buildings, forest, caves, terrain, urban canyons) Concealed routes Geo-political, religious, ethnic regions Probability of future attack Population density Civil unrest distributions Temporal/Semantic Analysis Associate individual IEDs with both low-level & abstracted features (both negative & positive correlation) Cluster IEDs based on similarity across these dimensions Evaluate clusters relative to temporal/semantic attributes for positive & negative trends/patterns Temporal order Time of day, day of week Relation to religious holidays (before, during, after) Relation to political events Relation to Blue Force convoy types, behavior, tactics VISE helps derive & visualize sensitivity to location, time of day, geo-politics, …

51 Example #3 (Optional) Interpreting Behavior - Intent Estimation Intent estimation requires perceived behavior be interpreted with respect to underlying (non-sensor-derived) context Hostile intent - weak indicators Direction and speed toward Blue Force elements Possible hide Concealed portions of trajectory Lightly-traveled road movement Low terrain-gradient off-road Night travel Object classification Hostile intent - possible strong indicators Object ID Avoids checkpoint Curfew violation Moderate terrain-gradient off-road Hide + concealed + lightly traveled + night + location + group size + apparent coordination +…

52 Detecting an IED Emplacement Operation Track vehicles to a high probability IED placement location Recognize stopped vehicles with high GMTI activity and no track development EO/IR confirmation of human activity Tracks leaving after requisite amount of time to emplace weapon

53 Dismount Tracking Example Road Dense foliage Government buildings Bus depot Trails Suspected strongholds HUMINT tip-off Near roads Near bus depot Application of context knowledge supports understanding

54 Overall objective Attack Village Bravo High-level goal: Cross river Mid-level goals: Reach bridge Lower-level goals: Overland route development: shortest path, lowest resistance,... X X Village Bravo (Likely target objective) Understanding Behavior Within a Global-to-Local Context

55 Top-Down, Global Reasoning River 1 Spatial Object Database Semantic Object Database Lines of Communication Rivers Roads Road 1 Road 2 River 1 River 2

56 Begin Seek Bridge 1 Hypothesis Management Road following strategy North of Lake 1 South of Lake 1 Closest road seeking strategy Minimum terrain gradient Maximum concealment Shortest distance Global strategy

57 South Lake 1 Begin Seek Bridge 1 Robust Hypothesis Pruning Instantiated Route: South of Lake 1 Global hypothesis Mid-level hypotheses Low-level hypotheses

58 VISE registration v. tie-point based approach (e.g., HART) Edge & areal-based v. intersecting edges Finds individual features Handles features with arbitrary shapes (not just rectilinear) Distinguishing VISE Characteristics Set operation support High performance vector Boolean operations Multi-resolution feature generalization Supports fuzzy point, line, and region features Generalized fuzzy membership functions Mixed Boolean and fuzzy set operations Search Top-down multiple level of abstraction/multiple resolution Generalized spatial windowing (not just bounding boxes) Database preserves all distance metrics for vector-based data Encourages the development of robust, context-sensitive algorithms Underlying database supports development of highly intuitive approaches Perceptual-based spatial similarity match (edge, area, XOR) Efficient reasoning across diverse problem dimensions (spatial & non-spatial)

59 End of Briefing

Download ppt "Situation Understanding & Threat Prediction R. Antony SAIC VISE Support to Kinematics + ID = Understanding KinematicsClassificationIDObjective/IntentThreatBehavior."

Similar presentations

Ads by Google