ORION Project-team Monique THONNAT INRIA Sophia Antipolis Creation: July 1995 Multidisciplinary team: artificial intelligence, software engineering, computer vision

Evaluation May 2006 Orion 2 Team Presentation Research Directions Cognitive Vision Reusable Systems Objectives for the next Period Contents

Evaluation May 2006 Orion 3 4 Research Scientists: François Bremond (CR1 Inria) Sabine Moisan (CR1 Inria, HDR) Annie Ressouche (CR1 Inria) (team leader) Monique Thonnat (DR1 Inria) 1 External Collaborator: Jean-Paul Rigault (Prof. UNSA Inria secondment) 4 Temporary Engineers: Etienne Corvee, Ruihua Ma, Valery Valentin, Thinh Van Vu 7 PhD Students: Bui Binh, Bernard Boulay, Naoufel Kayati, Le Thi Lan, Mohamed Becha Kaaniche, Vincent Martin, Marcos Zuniga Team presentation (May 2006)

Evaluation May 2006 Orion 4 Objective: Intelligent Reusable Systems for Cognitive Vision Cognitive Vision: Interpretation of static images Video understanding Reusable Systems: Program Supervision LAMA Software platform Research directions

Evaluation May 2006 Orion 5 Cognitive Vision: Image interpretation (ECVision European network on cognitive vision, EUCognition) vs. computer vision (INRIA CogB) Video understanding (USC Los Angeles, Georgia Tech. Atlanta, Univ. Central Florida, NUCK Taiwan, Univ. Kingston UK, INRIA Prima) Reusable Systems: Program supervision: e.g., scheduling (ASPEN and CASPER at JPL), image processing (Hermès at Univ. Caen, ExTI at IRIT)… Platform approach: e.g., ontology management (Protegé at Stanford), frameworks for multi agents (Aglets, Jade, Oasis at LIP6), distributed object community (Oasis at INRIA Sophia)… Orion team positioning

Evaluation May 2006 Orion 6 Objective: semantic interpretation of static 2D images Recognition of object categories (versus individuals) Recognition of scenes involving several objects with spatial reasoning Intelligent management of image processing programs Towards a cognitive vision platform Cognitive Vision : Image Interpretation

Evaluation May 2006 Orion 7 Scientific achievements: Knowledge acquisition: A visual concept ontology with 144 spatial, color and texture concepts [MVA04] Learning: Visual concept detectors [IVC06] Image segmentation parameters [ICVSa06] Cognitive vision platform Architecture [ICVS03] Object class recognition algorithm [CIVR05] Cognitive Vision : Image Interpretation

Evaluation May 2006 Orion 8 Self Assessment: Strong points: Visual concept ontology as user-friendly intermediate layer between image processing and application domain Automatic building of the visual concept detectors Still open issues: Learning for image segmentation Temporal visual concept ontology Cognitive Vision: Image Interpretation

Evaluation May 2006 Orion 9 Objective: Real time recognition of interesting behaviors How? Data captured by video surveillance cameras Original video understanding approach mixing: computer vision: 4D analysis (3D + temporal analysis) artificial intelligence: a priori knowledge (scenario, environment) software engineering: reusable VSIP platform Cognitive Vision: Video Understanding

Evaluation May 2006 Orion 10 Cognitive Vision: Video Understanding Segmentation Classification Tracking Scenario Recognition Alarms access to forbidden area 3D scene model Scenario models A priori Knowledge Objective: Interpretation of videos from pixels to alarms

Evaluation May 2006 Orion 11 Scientific achievements: Multi-sensor video understanding: 2 to 4 video cameras overlapping or not [IDSS03,JASP05] Video cameras + optical cells + contact sensors [AVSS05]… Learning: parameter tuning[MVAa06] frequent temporal scenarios models [ICVSb06] Temporal scenario: a new real time recognition algorithm [IJCAI03,ICVS03] a new representation language [MVAb06,ECAI02,KES02] Cognitive Vision: Video Understanding

Evaluation May 2006 Orion 12 Industrial impact: Strong impact in visual surveillance (metro station, bank agency, building access control, onboard train, airport) 4 European projects (ADVISOR, AVITRACK, SERKET, CARETAKER) 5 industrial contracts with RATP, ALSTOM, SNCF, Credit Agricole, STMicroelectronics 2 transfer activities with BULL (Paris), VIGITEC (Brussels) Creation of a start-up Keeneo July 2005 (8 persons) for industrialization and exploitation of VSIP library. Cognitive Vision: Video Understanding

Evaluation May 2006 Orion 13 Cognitive Vision: Video Understanding Intelligent video surveillance of Bank agencies

Toulouse - 3rd June 2004 Orion 14 “Unloading Global Operation” Cognitive Vision: Video Understanding

Toulouse - 3rd June 2004 Orion 15 Airport Apron Monitoring “Unloading Operation” European AVITRACK project Cognitive Vision: Video Understanding

Evaluation May 2006 Orion 16 Self Assessment: Strong points: Video understanding approach: real time, effective techniques used by external academic and industrial teams Launch of an evaluation competition for video surveillance algorithms (ETISEO) with currently 25 international teams Still open issues: Learning Multi sensor Cognitive Vision: Video Understanding

Evaluation May 2006 Orion 17 Reusable Systems: original approach for the reuse of programs with program supervision techniques Program supervision: Automate the (re)configuration and execution of programs selection, scheduling, execution, and control of results Knowledge-based approach: knowledge modeling, planning techniques, ….. Reusable Systems: Program Supervision

Evaluation May 2006 Orion 18 Reusable Systems: LAMA Platform Reusable Systems: Reuse of tools to design knowledge- based systems (KBS) LAMA Software Platform: Set of toolkits to facilitate design and evolution of KBS elements: engines, GUI, knowledge languages, learning and verification facilities… Software Engineering approach: genericity, frameworks, objects and components LAMA Problem Solving KBS provide generic components and tools raise new issues, to be abstracted into new components Virtuous Circle

Evaluation May 2006 Orion 19 Reusable Systems: LAMA Platform Task dedicated Engine Knowledge Base KBS User Expert Task dedicated GUI Task dedicated Language with compiler & KB verification Blocks Java graphic library for GUIs Verification library for knowledge bases Compilers/verifiers generators for knowledge description languages Framework for engine design & knowledge representation support and task specific layers LAMADesigner Program Supervision Object Recognition Model Calibration

Evaluation May 2006 Orion 20 Reusable Systems: Program Supervision Scientific achievements: Improvement of the Pegase engine (Pegase+) Multithreading, extensions to the YAKL language [ECAI02] Distributed program supervision Supervision Web server, multi-agent techniques, interoperability Pegase/Java/agents [TC06] Cooperation with image and video understanding Object recognition task using program supervision [ICTAI03] Interoperability with VSIP: program supervision for video understanding [ICVSc06]

Evaluation May 2006 Orion 21 Reusable Systems: LAMA Platform Scientific achievements: Enforcing LAMA safe usage Verification of LAMA component extensions relying on Model Checking approach [Informatica01, SEFM04] Encompassing new tasks Classification and object recognition in images: new engine and new knowledge representation language [ICTAI03] Model calibration in hydraulics: new engine/language (PhD co- directed with INPT and CEMAGREF) [KES03, JH05]

Evaluation May 2006 Orion 22 Reusable Systems: Self Assessment Strong points: Real time performance (Pegase+ and video) Using program supervision costs less than 5% of overall processing time LAMA genericity at work Different tasks (supervision, classification, calibration) in various application domains (hydraulics, biology, astronomy, video surveillance…) Shorter development time and safer code Reuse of concepts as well as code Several variants of a task sharing common concepts Extensibility and commitment to Standards

Evaluation May 2006 Orion 23 Creation of a new INRIA project-team PULSAR Perception Understanding and Learning Systems for Activity Recognition Theme: CogC Multimedia data: interpretation and man-machine interaction Multidisciplinary team: artificial intelligence, software engineering, computer vision Objective: Research on Cognitive Systems for Activity Recognition Focus on spatiotemporal activities of physical objects From sensor output to high level interpretation Objectives for the next period 1/5

Evaluation May 2006 Orion 24 PULSAR Scientific objectives: Two research axes: Scene Understanding for Activity Recognition Generic Components for Activity Recognition PULSAR Applications: Safety/security (e.g. intelligent surveillance) Healthcare (e.g. assistance to the elderly) Objectives for the next period 2/5

Evaluation May 2006 Orion 25 PULSAR: Scene Understanding for Activity Recognition Perception: multi-sensors, finer descriptors Understanding: uncertainty, 4D coherency, ontology for AR Learning: parameter setting, event detector, activity models, program supervision KB (risky objective) Objectives for the next period 3/5

Evaluation May 2006 Orion 26 PULSAR Generic Components for Activity Recognition From LAMA Platform to AR platform: Model extensions: modeling time and scenarios handling uncertainty User-friendliness and safeness of use: theory and tools for component frameworks scalability of verification methods Architecture improvement: parallelization, distribution, concurrence real time response domain specific software and graphical interface plugging Objectives for the next period 4/5

Evaluation May 2006 Orion 27 Short term objectives: Scene Understanding for Activity Recognition Perception: gesture analysis Understanding: ontology-based activity recognition uncertainty management Learning: primitive event detectors learning Generic Components for Activity Recognition Model of time and scenarios Internal concurrency and distributed architecture Objectives for the next period 5/5