1. SUMMARY DECLUTTERING CLUSTERINGACDC Outline  Automated Change Detection and Classification (ACDC) System  Computer-Aided Detection (CAD), Classification (CAC), Search (CAS), and Change Detection.  Clustering  NRL 6.2 FY05 New Start  Automated declutter mechanism for electronic displays  Summary

2. SUMMARY DECLUTTERING CLUSTERINGACDC  Ability to automatically detect / classify / identify objects in imagery and perform change detection  Current project: Side-scan imagery (SSI) for mine counter-measures (MCM)  Future/potential applications  Real-time Imagery in Cockpit (RTIC)  ECDIS  Weather / meteorological  Common Operational Picture (COP) ACDC

3. SUMMARY DECLUTTERING CLUSTERINGACDC Change Detection (using SSI) 1.Detect seafloor features (shadows, bright spots) 2.Classify detections (mines, rocks, sand waves, etc.) 3.Search historical database (position error) 4.Match new feature (to ideal features) 5.Perform area matching (uses clustering) 6.Identify features that don’t match: change detection ACDC

4. SUMMARY DECLUTTERING CLUSTERINGACDC MILECs found by:  Bright spots  Sizes  Shapes  Type of Mines  Shadows  Length (look angle)  Correct side  Proximity to bright spot Mine-Like Echoes (MILECs) Automatically detected in SSI Computer Aided Detection ACDC

5. SUMMARY DECLUTTERING CLUSTERINGACDC 000000000000000000000000000000 000000000000000000000000000000 000000000000000000000000000000 00000000000000000000000 00000010000000000000000 00000000000000000000000 00011100000000000000000 00000001111111111100000 00000000000000000000000 00001100000000000000000 00000001111111111100000 SSI stored in UNISIPS format as separate records (Lat/Lon, Altitude, Heading). Shadow Bitmap Thresholded using a hard limiting transfer function to find Shadows: a = func(n) | a = 1 if a < max_shadow_value Bright Spot Bitmap Thresholded using a hard limiting transfer function to find Bright Spots: a = func(n) | a = 1 if a > min_bright_spot_value Shadows & Bright Spots Marked Real-Time CAD ACDC

6. SUMMARY DECLUTTERING CLUSTERINGACDC CAD SNIPPET ADAPTIVE FILTERING AUTO-COMPLETE ACDC Computer Aided Classification

7. SUMMARY DECLUTTERING CLUSTERINGACDC (historical survey area) Historical SSI Database  Classification  Attributes  Imagery  Snippets  Features ACDC

8. SUMMARY DECLUTTERING CLUSTERINGACDC Vector Searchable Database (Handles position error!)  CAD / CAC new features (N) in areas where historical features (H) exist  Populate search database with H’s  Query search database for each N Historical: New: Spatial Query “ANDing” position error ellipses ACDC N = N 1, N 2, …, N n H = H 1, H 2, …, H n Results: N 1 = H 3 | H 10 H 10 H3H3 N1N1 H2H2

9. SUMMARY DECLUTTERING CLUSTERINGACDC Wavelet Networks for Feature Matching y(x,y) = Σ a sk w sk (x,y) a sk = wavelet coefficients w sk = basis functions Neural Network EXAMPLE: ACDC Training Set rectangle triangle circle unknown H 10 N1N1

10. SUMMARY DECLUTTERING CLUSTERINGACDC NEW Historical DatabaseNew Survey Data Cluster Region ACDC Wavelet Networks for Area Matching H2H2 H5H5 H 10 N4N4 N3N3 N2N2 N1N1 N1N1 N2N2 N3N3 N4N4

11. SUMMARY DECLUTTERING CLUSTERINGACDC Clustering for MCM  NRL has transitioned new algorithm to NAVO to cluster MILECs detected in SSI, in support of MCM efforts. I. Detect / classify / search for / identify mine-like objects (MILECs) in SSI. III. Smooth clusters, calculate density. 1 2 3 MILECsClutter / km 2 category x < 41 4 < x < 122 x > 123 II. Cluster objects into regions.

12. SUMMARY DECLUTTERING CLUSTERINGACDC Clustering algorithm  NRL algorithm clusters mine-like objects detected in SSI  NRL 7440.1 invention disclosed June 2003.  Uses geospatial bitmapping technique patented by Code 7440.1 in 2001 (U.S. Patent 6218965).  Unique method of clustering objects in 2D / 3D space: computationally efficient, single-pass, repeatable, operates on user-defined space, autonomous. DECLUTTERING

13. Longitude (X) Latitude (Y) Collection of points in geographic space (here, 2D)

14. Represent points as a geospatial bitmap. Each bit containing a point is “set;” all other bits are cleared. Bit size depends on scale.

15. “Grow” each set bit (representing each point) by setting the surrounding bits to form a predefined expansion shape.  Expansion shape dictated by data characteristics and user requirements.  Can use any shape that can be mathematically defined.  Size of expansion shape determines density of resultant clusters.

16. Points that are geographically close to each other will grow or cluster together. Result: new geospatial bitmaps.

17. Traverse each bitmap (in a consistent direction) to get vertices.

18. Smooth the bitmaps by dropping vertices if the resulting polygon still contains all the original points and has an area equal to or less than the unsmoothed bitmap.

19. Result of final iteration. Density of region = # original points / area of smoothed polygon.

20. SUMMARY DECLUTTERING CLUSTERINGACDC FY05 6.2 New Start  Internally funded by NRL 6.2 program (FY05-07)  After FY07, will need transition sponsors to implement in fleet systems (NGA / VVOD, NGA / DNC2, NAVAIR / TAMMAC, others)  Leveraging ongoing work from ACDC and other projects  Collaborating with Dr. Greg Trafton (Engineering Research Psychologist) DECLUTTERING

21. Clutter a confused multitude of objects Clutter as it pertains to this project: when additional information would result in performance degradation.

22. SUMMARY DECLUTTERING CLUSTERINGACDC The Clutter Problem  Our ability to collect data & “create” information is outpacing our ability to use and visualize the results. Clutter in all types of electronic displays is a massive and rapidly escalating problem.  Many researchers have documented link between increased clutter and degraded performance.  E.g., in cockpit displays, visual clutter can disrupt a pilot's visual attention, resulting in greater uncertainty concerning target locations. 1 1 Aretz (1988), Wickens (1993), Wickens & Carswell (1995) DECLUTTERING

23. SUMMARY DECLUTTERING CLUSTERINGACDC Problem (cont.)  Pilots want ability to declutter displays (e.g., driven by vector-based GIS-like databases) but such “flexibility means integration complexity and added pilot workload. Pilots should be flying, not building a map!" 1  "If the map display is too cluttered, I just turn it off!" 1  Automated decluttering requires good clutter metrics. Reasonably good clutter metrics exist for text displays, but not for graphical displays.  We propose to apply NRL detection / clustering algorithms and human factors principles to quantify clutter in electronic displays. 1 Quotes of F/A-18 pilots, from Lohrenz, et al. (1999) DECLUTTERING

24. SUMMARY DECLUTTERING CLUSTERINGACDC Relevance to COP charts imagery obstructions terrain NGA data Over the past decade, the COP has grown in functional complexity … (DISA, 1998) recon. routes troops targets C 4 I data DECLUTTERING

25. SUMMARY DECLUTTERING CLUSTERINGACDC Current declutter method Sample display downloaded from Nobeltec company website.website  Warfighter must manually remove an entire layer at once (brute-force filtering).  Need a more “intelligent” way to declutter electronic displays.  Warfighters and decision-makers should see all that is needed – but only what is needed – without extraneous data obscuring critical information. DECLUTTERING

26. SUMMARY DECLUTTERING CLUSTERINGACDC Measuring Clutter  Considerations:  Display type: aviation, meteorological, ECDIS, etc.  User expertise: novice vs. expert  Task: read-off, integrate, infer, working-memory  Incorporate established cognitive theory into new / enhanced clutter metrics:  “Global” vs. “local” information density  “Salience” (e.g., M. Zuschlag, 2004)  #Colors, color contrasts among adjacent objects  #Features of each “type” (points, lines, areas, text)  #Clusters of features, cluster density - NRL algorithm DECLUTTERING

27. SUMMARY DECLUTTERING CLUSTERINGACDC Clustering Algorithm  Expand technique to cluster objects in N x 2d (2d geospatial location + other attributes)  Primary challenges:  Mathematically define meaningful expansion shapes for feature layers and attributes.  Determine how clusters of various display feature types (points, lines, areas, text) interact with each other.  Apply established theories of human visual attention and search strategies to our methodology.  Bound the problem: focus on clustering features in NGA Vector Product Format (VPF) databases  Standard database for many DoD applications  Can be tailored with mission-specific data sets DECLUTTERING

28. SUMMARY DECLUTTERING CLUSTERINGACDC Validate Clutter Metrics  Can our metrics predict good display design (subjective / user preference)?  Interview technical experts  Compare clutter metrics with subjective evaluations  Can our metrics predict user performance and workload? Requires experimentation …  Independent variables Display type (aeronautical; nautical; meteorological; etc.) Clutter metric (uncluttered  very cluttered) User expertise (novice, expert) Task performed (read-off, integrate, infer, working memory, etc.)  Dependent variables Performance: time, accuracy, method/logic (e.g., w/ eye-tracker) Workload: subjective evaluation, secondary task, pupil dilation DECLUTTERING

29. SUMMARY DECLUTTERING CLUSTERINGACDC Summary  NRL developing ACDC system with algorithms to autonomously detect, classify, and cluster mine-like objects in SSI, and perform change detection via historical contact databases.  ACDC concepts/functions applicable to other types of imagery and objects  COP.  Detection and clustering algorithms will be exploited for new NRL project (FY05-07) to develop clutter metrics for electronic displays.  Will attempt to validate clutter metrics by comparing with measures of user performance and workload, and subjective evaluations of display design.  Results should be significant for future geospatial databases, db upgrades and display designs  COP. SUMMARY

30. Questions???