Download presentation
Presentation is loading. Please wait.
Published byJoelle Maddux Modified over 9 years ago
1
1 The Strategic Role of Vision in Cognitive Systems and Robotics - A Perspective from the EU Research Programme ECCV 2010, Special session on "Research Funding for Vision", September 9, 2010 Cécile Huet – cecile.huet@ec.europa.eu Unit E5 - Cognitive Systems, Interaction, Robotics DG Information Society and Media European Commission http://www.cognitivesystems.eu
2
2 Outline History Projects portfolio Perspectives and future opportunities Conclusion
3
3 Limitation of computer vision systems: Lack of ROBUSTNESS/VERSATILITY/SPEED Cognitive vision combine state-of-the-art image processing with cognitive reasoning: abstract representation, categorisation, memory organisation, self-adaptivity, contextual knowledge, behavioural/scene interpretation,… It all started with VISION: “Cognitive Vision” Initiative in 1998 (FP5): Rationale
4
4 Projects portfolio >140 projects - €480 million EU funding. FP5 (1998-2002) Cognitive vision - 9 projects FP6 (2002-2006) Cognitive Systems + Advanced robotics - 45 projects FP7 (2007-2013) Cognitive systems, Interaction, Robotics 68 running projects + 22 projects under negotiation
5
5 FP5 Cognitive Vision - achievements FP5 – DETECT Real Time Detection of Motion Picture Content in Live Broadcasts -> detect/track brand appearance in sport videos
6
6 FP5 Cognitive Vision - achievements FP5 – ACTIPRET Interpreting and Understanding Activities of Expert Operators for Teaching and Education Recognition of hand and arm gestures: original image with overlay and 3D representation Long term goal: giving hints on how to clear the paper jam in a copy machine
7
7 FP5 Cognitive Vision - achievements LAVA - Learning for adaptable visual assistants Categorization and interpretation of large numbers of objects, scenes and events, in real settings. Recognition/Categorization: Occlusion – cluttered background – viewpoint-rotation-scale,… invariance, lighting conditions,… Categorization - Cope with different forms of the same object (shape, colours, size, angle etc) and an open set of instances of the object Scene understanding – disambiguation –> multi cue/information fusion – context,… Source: www
8
8 FP6: Cognitive systems research - motivation Beyond vision…. Achieving naturalness, versatility, robustness at system level real-world from constrained to real-world environments general solutions from application-specific to general solutions systems approach from component methods to a systems approach working across disciplines from a ‘monodisciplinary’ research effort to working across disciplines RESULTS: Integration of perception (attention mechanisms, affordances, multi-sensory/cue…), representation, learning, reasoning, decision-making, action and communication, cooperation (HRI/RRI), anticipation,… Strong multidisciplinarity (neurosc., cog sci., engineering, AI,…)
9
9 Robotics (navigation – manipulation) - action-perception(COSPAL- SPARK), affordances (MACS-PACO-PLUS), anticipation(MINDRACES), reasoning - understanding (COSY), attention, knowledge-reasoning (GNOSYS), observation (MATHESIS), decision-making (DECISIONS-IN- MOTION), reasoning & self-preservation (ICEA), Joint Action (JAST), Active haptic(SENSOPAC), developmental (ROBOT-CUB) Interfaces - audio-visual- HHI/HRI(POP), HRI (PACO-PLUS, COSY), HRI- RRI (JAST) – ECAS (RASCALLI) Assistive technologies - Blind (DECISIONS-IN-MOTION- CASBLIP) – driving assistance (BACS) – Augmented human action (PACO-PLUS) Scene analysis/digital content - audio-visual + attention (POP-DIRAC), language & information discovery (RASCALLI), description image/video + text (CLASS), man-made scenes interpretation (eTrims), description of human behaviour(movement-facial expression) from video sequences (HERMES) + euCognition / PASCAL FP6: Cognitive systems projects - classification
10
10 HRI FEELIX GROWING: FEEL, Interact, eXpress: a Global appRoach to develOpment With INterdisciplinary Grounding INDIGO: Interaction with Personality and Dialogue Enabled Robots COMMROB: Advanced Behaviour and High-level Multimodal Communication With and Among Robots PHRIENDS: Physical Human-Robot Interaction: DepENDability and Safety ROBOT – FOR HUMAN ASSISTANCE (environments: Firefighter/Urban pedestrian area/home) VIEWFINDER: Vision and Chemiresistor Equipped Web-connected Finding Robots / GUARDIANS URUS: Ubiquitous Networking Robotics in Urban Settings Robots@home: An Open Platform for Home Robotics (platform-perception-navigation) SWARM – for human assistance or autonomous tasks IWARD: Intelligent Robot Swarm for Attendance, Recognition, Cleaning and Delivery (+HRI) ROBOSWARM: Knowledge Environment for Interacting ROBOt SWARMs (IT infrastr./Cleaning Hospital) GUARDIANS: Group of Unmanned Assistant Robots Deployed In Aggregative Navigation supported by Scent detection Network of Excellence / Coordination and Support Actions EURON: European Robotics Research Network CARE: Coordination Actions for Robotics in Europe RoSta: Robot Standards and Reference Architectures (+benchmarking) RAWSEEDS: Robotics Advancement through Web-publishing of Data Sets FP6: Advanced Robotics projects - classification
11
11 FP6 – Vision: Examples CASBLIP: Cognitive Aid System for Blind People Sensory substitution: from vision to audio information DECISIONS-IN-MOTION: Neural Decision-Making in Motion understand the neural mechanisms underlying fast target selection and sensory-guided decisions and actions in humans in motion transfer this knowledge into an artificial visual system for automated robotic navigation.
12
12 Challenges greatly improving robustness etc. requires rethinking the way systems are engineered -> theories are needed Scientific approach: understanding of what both natural and artificial systems can and cannot do, and how and why integration of disciplines: robotics, AI, computer vision, natural language, cognitive science, psychology,… mathematics,… philosophy of mind,… Equally importantly we need to be able to support claims that progress is being made identify real applications that would greatly benefit from such systems FP7 - Challenge 2: Cognitive Systems, Robotics, Interaction
13
13 HMI FP7 – CALL1 – PROJECTS CLASSIFICATION Cognition & Language emergence in robots Autonomous Robots Dexterous Manipulation HRI Perception & Sensors Scene understanding PROMETHEUS CoFRIEND SCOVIS Sensor network Spoken language Multimodal interaction Cognition for Human Assistance Machine Learning Medical – Rehab./Surgery SF SPARK II REPLICATOR CHIROPING EYESHOTS SEARISE DIPLECS CogX LIREC SEMAINE PINVIEW ALEAR ITALK ROSSI CLASSIC EMIME ROBOCAST MIMICS GRASP CHRIS DEXMART POETICON PASCAL 2
14
14 FP7 – CALL3 – PROJECTS CLASSIFICATION: SCENE UNDERSTANDING SCENE UNDERSTANDING NAVIGATION CA in Cog Sys URBAN MULTI-ROBOT / SWARM ACOUSTIC SCENE ANALYSIS UNDERWATERAERIAL HRI SCANDLE Co3AUVs SHOAL FILOSE EUROPA SFLY EUCOGII
15
15 FP7 – CALL3 – PROJECTS CLASSIFICATION FROM AN APPLICATION ORIENTED PERSPECTIVE IM_CLEVER HANDLE ECHORD ECCEROBOT MOBILE MANIPULATION SORTING OBJECTS ASSEMBLY SERVICE ROBOTICS & HRI EUROPA GUIDANCE DELIVERY TRANSPORT HUMOUR REHABILITATION MOTOR SKILL LEARNING STIFF PROSTHETICS ROBOSKIN PROGRAMMING BY DEMO/ TOUCH-BASED SOCIAL INTERACTION HUMANOBS SOCIAL- COMMUNICATIVE SKILLS (multimodal interaction – TV-Host) Co3AUVs SCANDLE MONITORING SURVEILLANCE SECURITY FILOSE SHOAL SFLY ENVIRONMENTAL MONITORING SEARCH & RESCUE BRICS ROSETTA INDUSTRY / MANUFACTURING
16
16 FP7 – CALL4 – PROJECTS CLASSIFICATION HRI AUDIO- VISUAL NAVIGATION BIO-INSPIRED LOCOMOTION MOTOR CONTROL DEXTEROUS MANIPULATION SENSOR NETWORK/ HUMAN ACTION RECO TRIDENT PERCEPTION: VISION ROBOSOM FIRST-MM COGNITO euROBOTICS CA in ROBOTICS VANAHEIM TACO GERT THE HUMAVIPS MULTIMODAL LANGUAGE NIFTI ALIZ-e IURO AMARSI WWW SHARED DB: Robots actions MASH ROBOEARTH HAPTICS + VISION AIROBOTS RADHAR GARNICS
17
17 Shared resources: Contribute to and benefit from: RoboEarth – Robots sharing a knowledge base for world modelling and learning of actions - Shared resources: Contribute to and benefit from: RoboEarth – Robots sharing a knowledge base for world modelling and learning of actions - http://www.roboearth.org/ World Wide Web for robots: a giant network and database repository where robots around the world access and continually update to share information and learn from each other about their behaviour and their environment. Improve any robot’s 3D sensing, acting and learning capabilities.
18
18 Shared resources: Contribute to and benefit from: MASH - Massive Sets of Heuristics for Machine Learning Design of complex learning systems integrating large and heterogeneous families of feature extraction modules.feature extraction Open web-based platform where contributors will find tools to interact and communicate with each other, and the means to integrate their feature extractors in continuously running experiments. Quantitative evaluation of submitted modules Performance will be measured on action selection with a robotic arm and in a simulated 3d environment, and object detection.action selectionobject detection http://mash-project.eu/
19
19 FP7 – CALL FoF – Smart Factories: ICT for agile and environmentally friendly manufacturing – Target outcome c) Robotics-enabled production processes tested and validated in real-world environment MANIPULATION/ HANDLING SORTING OBJECTS PRECISION FLEXIBILITY/SCALABILITY RELIABILITY ROBUSTNESS VERSATILITY MANIPULATION NON-RIGID / HEAVY OBJECTS PRECISION FLEXIBILITY/SCALABILITY RELIABILITY ROBUSTNESS VERSATILITY MANIPULATION NON-RIGID / HEAVY OBJECTS PACKAGING CustomPacker HRI + HR COOP.: WORKER DETECTION + INTENTION RECO- SKILL LEARNING PACKAGING ELECTRONIC CONSUMER GOODS CUSTOMPACKER NAVIGATION / PLANNING: MOBILE MANIPULATION / MOBILE ASSISANT: LOGISTICS + ASSEMBLY TAPAS VISUAL INSPECTION VISUAL SERVOING SHOE ROBOFOOT
20
20 VISION + Object/scene understanding FP7 – CALL6 – PROJECTS CLASSIFICATION (! Subject to successful negotiation) CA in ACS NAVIGATION DEXTEROUS MANIPULATION DEXTEROUS MANIPULATION HRI/SOCIAL INTERACT GOAL-DIRECTED/ANTICIPATION SENSOR NETWORK
21
21 INDICATIVE - BASED ON DRAFT TEXT ! INFORMATION TO BE CONFIRMED TARGET a): Research projects on Robotic systems operating in real-world environments: + functionalities + autonomy, safety, robustness, efficiency, and ease of use AS APPROPRIATE: + new materials and advanced sensor, actuator, effector, memory and control technologies. ->> validation through REAL-WORLD scenarios TYPE OF PROJECTS : STReP (Specific Targeted Research Projects) High risk/Focussing on specific research topic/component IP (Integrated Projects) System oriented BUDGET: 70M€ DEADLINE: 18.01.2011 What next? The 2010 Call for Proposals – FP7 - ICT Call 7 - http://www.cognitivesystems.eu
22
22 INDICATIVE - BASED ON DRAFT TEXT ! INFORMATION TO BE CONFIRMED TARGET d) Coordinated Action(CA) Fostering communication and co-operation between robotics and cognitive systems research communities: + knowledge sharing: EU, national, and international; + open source; R&D roadmaps, market potential, user acceptance, standardisation, education, ethics, socioeconomic impacts; outreach to relevant professional and general audiences. TYPE OF PROJECTS : CA BUDGET: 3M€ DEADLINE: 18.01.2011 What next? The 2010 Call for Proposals – FP7 - ICT Call 7 - http://www.cognitivesystems.eu
23
23 INDICATIVE - BASED ON DRAFT TEXT ! INFORMATION TO BE CONFIRMED Cognition and control in complex systems (IP and STReP) Cross-fertilisation between academic and industrial robotics research (IP) Targeted competitions for smarter robots (CA) INDICATIVE DEADLINE: 17.04.2012 What next? The 2011/12 Call for Proposals – FP7 - ICT Call 9 - http://www.cognitivesystems.eu
24
24 Perspectives Outcome so far and next steps New research questions -> need for further scientific progress Also technological progress and application motivated research, building on previous results – innovation in integration and implementation of results to improve robotics functionalities and features. Real world scenarios: source of research questions + validate research results Demonstrate progress: Benchmarking
25
25 Role of vision? CONCLUSION FP7 – Workprogramme text « Research and development […] will be guided by demanding, yet pragmatic application scenarios »
26
26 Critical!! CONCLUSION FP7 – Workprogramme text « Research and development […] will be guided by demanding, yet pragmatic application scenarios »
27
27 THANK YOU! http://www.cognitivesystems.eu
Similar presentations
© 2024 SlidePlayer.com Inc.
All rights reserved.