1. KEO Final presentation - Esrin 11/12/2008 1 Knowledge-centred Earth Observation Final presentation acs-keo@acsys.it

2. KEO Final presentation - Esrin 11/12/2008 2 ACS is a private Italian SW engineering company specialised in satellite remote sensing: Founded in 1979 Headquarters in Rome Research Centre in Matera (since 2001) 110 Employees (90% graduates, 11 PhDs) Quality System Certification ISO 9001 (since ‘99) Certified balance sheet since 1999 About ACS

3. KEO Final presentation - Esrin 11/12/2008 3 Integrated Systems Software solutions Services and Consultancies In the fields of: - Earth Observation Satellites Ground Segments - Environment Monitoring & Management Applications - Advanced & Immersive data visualisation Technology transfer Medical Applications ACS proposes

4. KEO Final presentation - Esrin 11/12/2008 4 Ground Segment Technologies Payload Ground Segments Processors (SAR and Optical) Data Mining Mission Planning Tools User Services ACS Capabilities

5. KEO Final presentation - Esrin 11/12/2008 5 EO Applications Vegetation Monitoring Detection of Marine Pollution Volcano Eruption Monitoring Environmental Risk Monitoring Cartography and DEM/DTM Desertification Monitoring Urban Expansion Applied Interferometry ACS Capabilities

6. KEO Final presentation - Esrin 11/12/2008 6 ACS Systems in the World

7. KEO Final presentation - Esrin 11/12/2008 7 Knowledge-based Information Mining (KIM) tool –Interactive image collection analysis using Probabilistic Information Mining –Cooperation of ACS, DLR, ETHZ –Technology development started in 2001 (ESA TRP) and improved over time Knowledge-centred Earth Observation (KEO) –KIM + rich, graphic, distributed environment for automated Information Extraction –Cooperation of ACS, DLR, CNES, GTD –Started in 2005 (ESA TRP) Image Information Mining - Time Series (IIM-TS) –KEO modules for co-registration and Time Series handling –Cooperation of ACS, CNES, DLR, IGUASSU, JRC, MEEO, SARMAP, UNIPV, UNITN, VTT –Started in 2007 (ESA TRP) ACS & IIM

8. KEO Final presentation - Esrin 11/12/2008 8 The System

9. KEO Final presentation - Esrin 11/12/2008 9 What is KEO ? Distributed Component-based Processing Environment Graphically build of Processing Components Image Information Mining (IIM) Processing with Interactive Training Processing results available through OGC services (WMS, WFS, …) Processing graphs and/or processing results are easy to publish in SSE

10. KEO Final presentation - Esrin 11/12/2008 10 KEO SW architecture

11. KEO Final presentation - Esrin 11/12/2008 11 KEO HW Architecture

12. KEO Final presentation - Esrin 11/12/2008 12 FP HW/SW architecture

13. KEO Final presentation - Esrin 11/12/2008 13 KEO Users Domain Expert –Not an expert in remote sensing –Needs information to make decision Remote Sensing Expert –A scientist with signal & image processing know-how –Able to build chains of processing components

14. KEO Final presentation - Esrin 11/12/2008 14 Live Demos

15. KEO Final presentation - Esrin 11/12/2008 15 KEO Users Interfaces SSE [ Domain Expert ] http://keo.esrin.esa.int KAOS [ Domain & Remote Sensing Experts ] –Server Monitor –Image Browser –Image Info Viewer –Product Explorer –World Wind Viewer –FEP Designer

16. KEO Final presentation - Esrin 11/12/2008 16 Live Demo Glossary FEP ( Feature Extraction Processor ) FEP Engine FEP Actuator Data Flow Module [CLI, JAVA, WS] Ports Connection (Semantic Check)

17. KEO Final presentation - Esrin 11/12/2008 17 Live Demo NDVI 1 Problem –Vegetation Detection using NDVI Solution –Search in KEO a NDVI modules Action –Run the found FEP

18. KEO Final presentation - Esrin 11/12/2008 18 Live Demo NDVI 2 Problem –NDVI isn’t good enough Solution –Search in KEO for different NDVI modules Action –Create a new FEP with the found CLI module and run it

19. KEO Final presentation - Esrin 11/12/2008 19 Live Demo NDVI classification Problem –Classify the NDVI image to find an easy way to detect a particular type of vegetation Solution –Use an available classification algorithm Action –Create a new CLI with the classifier and integrate it within our FEP

20. KEO Final presentation - Esrin 11/12/2008 20 Knowledge-based Information Mining KIM Live Demo KIM subsystem

21. KEO Final presentation - Esrin 11/12/2008 21 Live Demo Motivation KEO/FEP: a tool for large-scale algorithmic processing –For well-specified phenomena of interest –With well-known solutions KEO/KIM: a tool for Exploratory Analysis –For unknown / hard-to-specify phenomena –Via examples [using color, shape, multi-scale texture for optical, radar sensors] –Interactive KEO processor definition

22. KEO Final presentation - Esrin 11/12/2008 22 Live Demo Image Browser An image archive client: –browse by quicklook visualization –search by attributes (e.g. date, footprint...) –search by content!

23. KEO Final presentation - Esrin 11/12/2008 23 Live Demo Label Designer A rich interaction environment Iterative process: –YES/NO examples via mouse clicks –interface reacts: red interest areas Zoom levels, undo, real-time enhancement... YES NO

24. KEO Final presentation - Esrin 11/12/2008 24 Live Demo Label Designer Fast, iterative supervised training across archive Can be re-used, refined, extended, managed Associated to interactively user-defined textual tags Seamless data exchange with archive

25. KEO Final presentation - Esrin 11/12/2008 25 Live Demo Thematic Maps User-defined interest rapid mapping Automatic production on-the-fly for the entire archive Select + rank relevant images by content Can be published on SSE or WCS/WMS

26. KEO Final presentation - Esrin 11/12/2008 26 Live Demo Product Explorer Expert exploratory analysis Define new product format combining –Thematic maps –Primitive signals for further analysis / other users

27. KEO Final presentation - Esrin 11/12/2008 27 KIM: image search engine Application- independent descriptors Content simplifier Interactive sub-space selection + tagging by labels Google, Yahoo Crawler - data collection / analysis Content indexer Interactive front end / ranking “ingestion” Evolution

28. KEO Final presentation - Esrin 11/12/2008 28 System Architecture

29. KEO Final presentation - Esrin 11/12/2008 29 Main componets Ingestion subsystem FEP subsystem KARISMA application Reference DataSet Ontology subsystem OGC services KAOS client application

30. KEO Final presentation - Esrin 11/12/2008 30 Ingestion subsystem In charge of ingesting products extracted and/or computing: information for KIM processing features and KIM data feature labels and labels maps co-registered products

31. KEO Final presentation - Esrin 11/12/2008 31 FEP subsystem A data-flow engine in charge of extracting features using Feature Extraction Processors made of: Java-coded modules CLI programs external Web Services

32. KEO Final presentation - Esrin 11/12/2008 32 FEP subsystem Made of: Designer, used to graphically draw a processing graph Engine, the executor and coordinator of the processing graph Actuator, the executor of the platform-dependent modules

33. KEO Final presentation - Esrin 11/12/2008 33 FEP Engine

34. KEO Final presentation - Esrin 11/12/2008 34 FEP Actuator

35. KEO Final presentation - Esrin 11/12/2008 35 KARISMA The KEO back end application for KIM –Provides access to image archive –Provides access to KIM data –Implements most of the KIM engine –Manages users, projects, … J2EE application and some W/S Deployed on a glassfish application server

36. KEO Final presentation - Esrin 11/12/2008 36 KAOS KEO client application –KIM client [ KARISMA ] –FEP client [ KAFE ] –KEO system management

37. KEO Final presentation - Esrin 11/12/2008 37 AIR web Service Automatic Image Registration –First level of integration of GDAL, but still semi-automatic –Configuration files added for processing different image pairs –Command Line Interface simplification and logfiles centralization Deformation Model Management System –XML files based on OGC standards –C++ library for handling a Deformation Model –Test and visualization tools –Some anomalies still to be corrected Web Service (AIR-WS) –AIR-WS Java application developed on top of AIR Python scripts –Test and visualization tools

38. KEO Final presentation - Esrin 11/12/2008 38 Ontology subsystem Free text search inside KEO by KAOS client

39. KEO Final presentation - Esrin 11/12/2008 39 Reference Data Set Set of reference data used to test and validate FEPs Collection of georeferenced Images and any other relevant data (text, pdf, movie, URLs, …) Based on GeoNetwork

40. KEO Final presentation - Esrin 11/12/2008 40 GeoNetwork Features Users Search in multiple catalogues through one website Access data from source site File download Data visualization through web map viewer Data Providers Standardized process to document and publish Online metadata editor Different metadata standards (ISO 19115, ISO 19139, FGDC, Doublin Core) Different sharing / security levels File upload

41. KEO Final presentation - Esrin 11/12/2008 41 GeoNetwork Features GeoNetwork Technology Java language (platform independent) Servlet environment (Tomcat, Jetty,...) XML data representation (JDOM) XSL templates for presentation (XALAN) JDBC for database access (runs on any compliant DBMS) XML Schema for metadata editing Catalogue interoperability Full OGC Catalog v2.0 support ISO profile (Reference OGC implementatio) Open Archive Initiative support (OAI-MPH) RSS and GeoRSS newsfeeds OpenSearch with GEO extension Metadata harvesting & synchronization

42. KEO Final presentation - Esrin 11/12/2008 42 OGC services WMS and WFS for data extracted by PIM Batch processors Based on UMN MapServer WMS and WFS exposed to SSE

43. KEO Final presentation - Esrin 11/12/2008 43 System Evolutions

44. KEO Final presentation - Esrin 11/12/2008 44 Technology and Research continues with IIMTS project aimed at extending KEO and KIM capabilities to Image Time Series with the purpose of: –Identifying and locating changes –Identifying and locating patterns –Tracking and following movement of recurring features which could represent potentially useful information for specific applications: –Urban areas monitoring –Agricultural monitoring –Emergency and Security Further Evolutions in IIM

45. KEO Final presentation - Esrin 11/12/2008 45 KEO provides a distributed environment where Feature Extraction Algorithms (including Time Series) can be plugged and used as simple CLI modules (in any programming language). Any KEO registered user can deploy his own modules and make these modules usable by others thus optimising development efforts By design CLI modules run on processing nodes that can even be physically located on remote sites thus providing an effective solution for a distributed exploitation and processing environment ……………….. And in future ? Conclusions and Future

46. KEO Final presentation - Esrin 11/12/2008 46 Classification of Optical Images (including Very High resolution) Tools for Oil Spill and Ship Detection from SAR images Binary and Multiclass Change Detection tools from Optical and SAR Images Shape Change Detection Tools to point out geometric differences between regions from previous classifications Trend Analysis in Temporal Time Series also as an extension of the KIM concept. From May 2009 KEO will include Processing Components (developed within the Image Information Mining – Time Series project) in support to: Conclusions and Future

47. KEO Final presentation - Esrin 11/12/2008 47 A KEO system will be installed in Romania in the frame of the ESA- ROSA (Romanian Space Agency) agreement. ACS continues to study and prototype a second generation of KIM Systems A discussion with ESA is undergoing to define a working plan for KIM/KEO enhancements An assessment of what should come first in terms of evolutions is needed, so feedback from users would be highly appreciated ……….. ………….. What will happen in ‘09 ? Conclusions and Future

48. KEO Final presentation - Esrin 11/12/2008 48 People interested in becoming registered KEO users are invited to contact ESA! http://keo.esrin.esa.int For any question please contact: –ESA for KEO: Michele Iapaolomichele.iapaolo@esa.intmichele.iapaolo@esa.int –ESA for RDSs: Andrea Della Vecchiaandrea.della.vecchia@esa.intandrea.della.vecchia@esa.int –ACS: KEO Team acs-keo@acsys.itacs-keo@acsys.it Thank you for your Attention Conclusions and Future

49. KEO Final presentation - Esrin 11/12/2008 49 Questions?