1. Cognitive Colonization The Robotics Institute Carnegie Mellon University Bernardine Dias, Bruce Digney, Martial Hebert, Bart Nabbe, Tony Stentz, Scott Thayer

2. Presentation Outline  Requirements  Software Architecture  Perception and Mapping  Communal Learning  Robot Test Bed  Status and Future Work

3. Requirements  is robust to individual robot failure;  does not depend on reliable communications;  can perform global tasks given the limited sensing and computational capabilities of individual robots;  learn to perform better through experience. Distributed robotics for small-scale mobile robots calls for a software system that:

4. Cognitive Colonization Paradigm  dynamically assigning robots to tasks and checkpointing data;  treating communication as an opportunistic resource;  aggregating resources by distributing the computational and perceptual load across the group of robots;  sharing learned behaviors (both individual and group) between all robots. The proposed software system addresses these requirements by:

5. Software Architecture Models Centralized Distributed optimal intractable brittle sluggish communication heavy suboptimal tractable robust nimble communication light

6. Free Market Architecture  Robots in a team are organized as an economy  Team mission is best achieved when the economy maximizes production and minimizes costs  Robots interact with each other to exchange money for tasks to maximize profit  Robots are both self-interested and benevolent, since it is in their self interest to do global good

7. Simple Reasoning Robot 1 Robot 2 Task A = 120 Task B = 180 50 75 110 100 Robot 1 profit = 20 Robot 2 profit = 30

8. More Complex Reasoning Robot 1 Robot 2 Task A = 120 Task B = 180 50 75 110 100 60 Subcontract: (150 + 110) / 2 = 130 Robot 1 profit: 40 (20) Robot 2 profit: 50 (30)

9. Distributed Mapping Example Operator Exec <-- Revenue paid Tasks performed -->

10. Distributed Mapping Roles Unattached Robot Single Robot Command Unit Mapping Squad Mapping Squad Communications Squad

11. Architectural Framework Roles Resources Negotiations Locomotor Sensors CPU Mapper Comm Leader Exec Radio

12. Simple Mapping Example InitialFinal

13. Complex Mapping Example Initial Final

14. Simulated Mapping X X X X X X X X X X X X X X X X X X X

15. Simulator Movie

16. Simulated Mapping X X X X X X X X X X X X X X X X X X X

17. Architecture Features  Revenue, cost and profit  Negotiation and price  Competition vs. cooperation  Role determined via comparative advantage  Self organization  Learning and adaptation

18. Map Reconstruction Objectives  Reconstruction of 3-D map from multiple robots  Unknown or imprecise relative position  Recovery of positions and structure  Map reconstruction for operators Robot A Robot B

19. Typical Environment

20. Map Reconstruction: Approach  Approach:  Feature extraction  Initial feature matches  Recovery of epipolar geometry  Filtering of matches by re-projection  Recovery of motion and 3D structure

21. Feature Matching and Depth Recovery

22. Feature Map

23. Feature Matching and Depth Recovery

24. Feature Map

25. Communal Learning

26. Robot Death and Sacrifice  Quickly learning causes of robot death required for colony survival  ‘Buddy System’ used to preserve fatal situations and actions  When robot sacrifice is required maximal cause of death information will be extracted  Causes of death are high value commodities and quickly disseminated through the colony

27. Robots in Action

28. Current Status  Five working robot test beds with navigation, obstacle avoidance, point-to-point communication, and image streaming  First version of software architecture working for distributed coverage tasks  First version of cooperative stereo implemented with automatic feature selection  Prototype colony interface designed

29. Next Steps  Port the architecture to the real robots  Extend the architecture to support all robot roles needed for distributed mapping  Add learning for behavior parameters and transaction confidences  Perform map integration from multiple sensing points