CLARAty: Coupled Layer Architecture for Robotic Autonomy & CLARAty on FIDO Issa A.D. Nesnas Ames Research Center Carnegie Mellon University Jet Propulsion Laboratory October 25, 2002 Mars Technology Program Year-End Review FY02

October 25, 2002 CLARAty FY02 Review- I.A.N. 2 Presentation Overview Quad Chart Background and Overview Relevance to the missions CLARAty Team and Collaborators Level I & II Milestones Accomplishments –CLARAty running on FIDO –CLARAty Software Package developments –Distributed development process and testbed Future Plans Publications and website

October 25, 2002 CLARAty FY02 Review- I.A.N. 3 Objective Develop a unified and reusable robot control software that facilitates the integration of new technologies on various robotic and rovers platforms Milestones – FY01 through FY05 Prototype robotic software and implement on various rover platforms Integrate and demonstrate technology components from MTP RMSA program Demonstrate low-, mid- and high-level autonomy CLARAty: Coupled Layer Architecture for Robotic Autonomy Schedule and Funding Task Manager: Issa A.D. Nesnas, JPL (818) , jpl.nasa.gov Participating Organizations: JPL NASA Ames Research Center Carnegie Mellon University Facilities: Rocky 8, Rocky 7, K9, FIDO, and CLARAty testbed ROAMS, WITS JPL Mars Yard ActivityFY00FY01FY02FY03FY04FY05 Design CLARAty and implement path planning in it. Basic rover functionality integrated with decision capabilities Sophisticated rover functionality for Long Traverses Sophisticated rover functionality for Instrument Placement Autonoumous science and Health monitoring

October 25, 2002 CLARAty FY02 Review- I.A.N. 4 Background & Objectives Why are we doing this work? To capture and preserve robotics expertise from JPL and other centers To provide a framework for future NASA rover technology development and integration To reduce the cost of integrating new technologies To operate various robots from a unified framework To eliminate cost of redeveloping basic robotic infrastructure

October 25, 2002 CLARAty FY02 Review- I.A.N. 5 What is CLARAty? CLARAty is a unified and reusable software that provides robotic functionality and simplifies the integration of new technologies on robotic platforms

October 25, 2002 CLARAty FY02 Review- I.A.N. 6 Relevance to the Missions Why is this work relevant to the missions? Provides a common environment for development, test, and comparison of advanced robotic technologies Provides an infusion path for robotics technologies into flight missions Demonstrates technologies on relevant robotic systems Makes research rovers viable test platforms for flight algorithms (e.g. navigation) Is robust to changes in rover hardware designs Can be easily adapted to new research and flight prototype rovers (for ground-based testing)

October 25, 2002 CLARAty FY02 Review- I.A.N. 7 Measuring Success or Failure We succeed IF we: Significantly reduce integration time of new technology software onto real robotic systems Support multiple platforms with different hardware architectures Provide a service that is enabling for technologists Simplify the development/integrate/debug/test cycle for current and next generation NASA rovers Have people other than the developers using and “like” the system

October 25, 2002 CLARAty FY02 Review- I.A.N. 8 CLARAty Team CLARAty/Rocky8/Rocky7 –Max Bajracharya (34) –Edward Barlow (34) –Caroline Chouinard (36) –Gene Chalfant (34) –Hari Das (34) –Tara Estlin (36) –Dan Gaines (36) –Mehran Gangianpour (34) –Dan Helmick (34) –Won Soo Kim (34) –Michael Mossey (31) –Issa A.D. Nesnas (34) (Task Manager) –Ashitey Trebi-Ollennu (35/34) –Richard Petras (34) (Rocky lead) –Stergios Roumeliotis (Univ. of Minnesota) –Kevin Watson (34) NASA Ames Research Center –Maria Bualat –Clay Kunz –Randy Sargent –Anne Wright (lead) Carnegie Mellon University –Kam Lasater –Reid Simmons (lead) –Chris Urmson –David Wettergreen –Andy Yang Jet Propulsion Laboratory CLARAty on FIDO Team –Hrand Aghazarian (35/34) –Terrance Huntsberger (35/34) –Chris Leger (35/34) –Matthew Robinson (35/34) Five Summer Students

October 25, 2002 CLARAty FY02 Review- I.A.N. 9 CLARAty Collaborations (1/2) JPL - MDS Team –Participated with MDS team on CLARAty/MDS infusion task –Supported MDS by upgrading Rocky 7 to MSL-like hardware and provided working version of CLARAty software for migration to MDS JPL - ROAMS Team (A. Jain) –Close collaborations for the integration of the Functional and Decision Layers to the ROAM rover simulation JPL - CLEaR Team (F. Fisher) –Provided an instantiation of the CLARAty Decision Layer, integrated, tested and debugged on Rocky 7 and Rocky 8 rovers JPL - Instrument Simulation Team (M. Lee) –Worked closely with instrument simulation team to develop spectrometer and spectra infrastructure for CLARAty.

October 25, 2002 CLARAty FY02 Review- I.A.N. 10 CLARAty Collaborations (2/2) RMSA Teams: –ARC - Science Analysis Team (T. Roush) Close interactions to support integration of science analysis into the CLARAty environment. –JPL - Slope Navigation (L. Matthies) Collaboration for integration of visual odometry on rover platforms –University of Washington (C. Olson & R. Li) Close collaboration for integration of bundle adjustment and wide baseline stereo –Carnegie Mellon University (A. Stentz) Integration of D* path planner for long range traverse –Ames Research Center (E. Bandari) Integration of visual tracking ALERT Team (former REE) –Providing CLARAty communication and rover control infra-structure to REE for a navigation demonstration on Rocky 8

October 25, 2002 CLARAty FY02 Review- I.A.N. 11 Level I: Demonstrate long range traverse using the integration of global path planning and local navigation capabilities provided through the CLARAty architecture (09/2002) Level I: Demonstrate a CLARAty capability such as vision-based navigation (using the GESTALT navigator) onto the FIDO rover platform Level II: Demonstrate mission-like scenario for visiting multiple science targets using improved position estimation Demonstrate CLARAty locomotion functionality by interfacing to ROAMS at the motor control Milestones for FY02

Accomplishments

THE DECISION LAYER: Declarative model-based Mission and system constraints Global planning CLARAty = Coupled Layer Architecture for Robotic Autonomy INTERFACE: Access to various levels Commanding and updates THE FUNCTIONAL LAYER: Object-oriented abstractions Autonomous behavior Basic system functionality A Two-Layered Architecture Adaptation to a system

October 25, 2002 CLARAty FY02 Review- I.A.N. 14 General ITAR document complete and submitted to NASA HQ for submission to State Department Setup several working groups with flexible signup Upgraded and consolidate all VxWorks licenses (6 seats with full tool suite support) Setup a repository for 3rd software CLARAty depends on (ACE, Perl, Qt, etc.) Multiple team members can administer CLARAty site Established web-based operations (testbed, meetings, plans, procurements) CLARAty moved to using ACE for OS portability CLARAty supports several systems: VxWorks, Linux, Solaris, Mac X

October 25, 2002 CLARAty FY02 Review- I.A.N. 15 Solaris x86 Ames JPL Rocky 8 x86 VxWorks Linux Rocky 7 K9 ROAMS Currently Supported Platforms FIDO CMU Linux ATRV x86 JPL FIDO VxWorks x86 JPL ppc VxWorks JPL Linux

Challenges in Interoperability

October 25, 2002 CLARAty FY02 Review- I.A.N. 17 Distributed Hardware Architecture x86 Arch Wireless ethernet 1394 FireWire I2C Bus Rocky 8 Compact PCI Widgets  Single Axis Controllers  Current Sensing  Digital I/O  Analog I/O Actuator/Encoder s Potentiometers I2C Serial Bus 1394 Bus IMU RS232 Serial Sun Sensor

October 25, 2002 CLARAty FY02 Review- I.A.N. 18 Custom Architecture/Variability m68k Arch Framegrabbers Digital I/O Analog I/O Wireless ethernet Rocky 7 VME Arch Actuator/Encoders Potentiometers Parallel Custom Interface MUX/Handshaking Video Switcher Gyros Accels AIO PID Controllers

October 25, 2002 CLARAty FY02 Review- I.A.N. 19 Centralized Hardware Mapped Architecture x86 Arch Framegrabbers Digital I/O Analog I/O Wireless ethernet Fido PC104 Actuator/Encoders Potentiometers PID Control in Software Video Switcher IMU RS232 Serial

October 25, 2002 CLARAty FY02 Review- I.A.N. 20 CLARAty on FIDO Adaptation Controlled_Motor_Impl ControlledMotor Linear_Axis Joint Non-Resuable Layer R8_Motor Fido_MotorSim_Motor Trajectory Trajectory_Generator Mz HCTL_Chip Widget_Board Widget_Motor PID ControllerCounter DIO Analog Out Analog In MSI P460 MSI P430 MSI P415 MSI P430 Non-Resuable Resuable

October 25, 2002 CLARAty FY02 Review- I.A.N. 21 Code Reusability

October 25, 2002 CLARAty FY02 Review- I.A.N. 22 Simulation Hardware Drivers Adapting to a Rover Generic Functional Layer Rocky 8 Specialized Classes & Objects Connector Decision Layer Multi-level access Connector Rocky 8 Models/ Heuristics

October 25, 2002 CLARAty FY02 Review- I.A.N. 23 The Decision Layer Executives (e.g. TDL) General Planners (e.g. CASPER) Activity Database Rover Models FL Interface Plans

October 25, 2002 CLARAty FY02 Review- I.A.N. 24 The Functional Layer Transforms Motion Control VisionEstimation Input/Output Manipulation Navigation Communication Math Hardware Drivers Locomotion Rover Behaviors Path Planning Rocky 8 Rocky 7 K9 FIDO Science Simulation Sensor Adaptations Base Thickness indicates work done in area in FY02 Primarily in CLARAty on FIDO

CLARAty Software Packages

October 25, 2002 CLARAty FY02 Review- I.A.N. 26 Base Package Developed Generic Physical Component Base classes Math and Data Structure modules –Merging of array/matrix development in Ames branch with the main branch. Added support for sub-arrays, and STL iterators Bits and I/O modules (JPL) –New implementation of bits, digital and analog I/O modules to be more consistent with the rest of CLARAty, to provide support for reentrancy and to increase efficiency Telemetry module (ARC) –Added telemetry modules to represent data and parameter sets, handles general serialization and deserialization, provide multi-threaded clients, and increase efficiency Device module (ARC) –Provide base classes for all devices such as cameras, arms, masts, locomotors, etc. Uses telemetry classes and provides generic interfaces for accessing telemetry from any device in a consistent manner Power System module (ARC) –Provides information on devices such as power sources, batteries, and battery chargers. Implemented specializations for K9 power hardware

October 25, 2002 CLARAty FY02 Review- I.A.N. 27 Developed algorithms for partially- steered and fully-steered vehicles Developed continuous driving capability (JPL) Demonstrated on Rocky8 & Rocky7, and currently on FIDO (JPL) Third generation redesign (CMU) –Separated locomotor model from control –Added concept for wheel, steerable wheel, drive cmds, and drive sequences. –Adapting to Rocky 8, Rocky7, ATRV, FIDO, and K9 –Tested on Rocky 8, Rocky 7 and simulation (including ROAMS) Locomotion Package Front x y z C (d) Partially Steerable (e.g. Rocky 7) Front x y z C (e) All wheel steering (e.g. Rocky8, Fido, K9) (c) Steerable Axle (e.g.Hyperion) Continuous Driving and Separated Model from Control

October 25, 2002 CLARAty FY02 Review- I.A.N. 28 Example: collaborative development for locomotor JPL CMU Future ARC Designed for Rocky 7 Used Motor class Separated wheel control from locomotion Built-in pose estimation Generalized design for wheeled locomotors Full and partially steerable vehicle Used generic motor classes Implements fixed axle model Developed continuous driving Adapted to Rocky 8, Rocky 7, and Sim Version 1.0 Version 2.0 Separated model from control Add separate locomotor state Add concept of wheel and steerable wheel, Drive Cmd, Drive Sequence Adapt to ATRV, Sim, Rocky 7, Rocky 8 Version 3.0 Redesign/ mature Version 4.0 Use device and telemetry infrastructure Add adaptation to K9 Add JPL

October 25, 2002 CLARAty FY02 Review- I.A.N. 29 Vision Package Developed generic infrastructure for vision –Generic Camera, Stereo Camera, Camera Models, Image, Image operations (e.g. rectification, edge and corner detection), Image I/O, and Image transport Provided wrappers for vision algorithms –JPL Stereo, CAHVOR models, ARC Stereo, SVS Stereo, SLOG tracker Adapted package to various rovers –Rocky 8, K9, FIDO, Rocky 7 are currently using the same vision infrastucture Worked with Machine Vision Group to support integration visual odometry into CLARAty Worked with RMSA team (Olson, Roush), ARC, CMU, M. Lee in the development of this package Developed generic Image and Camera infrastructure

October 25, 2002 CLARAty FY02 Review- I.A.N. 30 Completed and tested FIDO-based EKF port to CLARAty (JPL) –Two tier Kalman Filter –Estimating heading (IMU + odometry), x, and y (odometry) Created second generation design for the Estimator module (JPL & Univ. of Minnesota) –Handle non-EKF based estimators –Designed to better integrate with Locomotor and other model infrastructure –Assume filter primary functions are to propagate state and update measurements Estimation Package FIDO EKF now fully integrated and tested in CLARAty

October 25, 2002 CLARAty FY02 Review- I.A.N. 31 Navigation & Mapper Package New modular design for the Navigator (CMU) –Based on generic action-selector - can be adapted to support navigation technologies –Specialized action selector to Map-based selector –Tightly integrated local and global cost functions (D*) –Navigator interfaces with Locomotor, Location Estimator, and Terrain Map generator. Refactored Gestalt to separate terrain evaluation from action selection (JPL) Developed infrastructure to process multi-tiered panoramic images for terrain evaluation (JPL) Acquired complete 3-D map of Mars Yard Developed Generic Navigation Infrastructure

October 25, 2002 CLARAty FY02 Review- I.A.N. 32 Demonstration of CLARAty Navigation Infrastructure in Simulation (CMU) DStar Map (final state) global cost function Morphin Terrain Analysis (scene from simulator)

October 25, 2002 CLARAty FY02 Review- I.A.N. 33 Resurrected dexter platform and made part of CLARAty testbed (JPL) Some upgrades to manipulation package to support acquisition of multi-tiered panoramic images (JPL) Manipulation Package Dexter Dexter Manipulation platform now part of CLARAty testbed

October 25, 2002 CLARAty FY02 Review- I.A.N. 34 Developed adaptation of the CLARAty controlled motor classes to interface with ROAMS simulation (JPL) Tested Locomotor sending commands to wheels which in, turn, send commands to controlled motors (JPL) Specialized Sim_Motors turn commands into streams for socket- based communication Drove Simulated Rocky 8 rover with new locomotor (CMU) Interface between DL/FL (rover-level and ROAMS) is also available Simulation Package CLARAty Locomotor talking to ROAMS via motor cmds

October 25, 2002 CLARAty FY02 Review- I.A.N. 35 Continued interactions with T. Roush’s team for integration of science analysis and spectrometer functionality in CLARAty Analysis code checked in the repository: –Carbonate Analysis –Edge Layer detection Continued interactions with Meemong Lee’s for interface with science instruments, analysis, and simulation. Code available in repository –Calibrated Reflectance Spectrum –Uncalibrated Spectrum Science Package Infrastructure for Spectrometer and terrain analysis

October 25, 2002 CLARAty FY02 Review- I.A.N. 36 Decision Layer Package Integrated D* Path Planner with Decision Layer Developed infrastructure for sharing terrain map with FL Migrating a version of CLEaR (CASPER/TDL) into CLARAty repository. Now available are: –TCM - Task Control Management used by TDL –Tangent Graph Path Planner (as used by DL) –D* Path Planner (as used by DL) –TBD: TDL and Java utilities Unifying DL dependencies to use the CLARAty modules (e.g. path planners - tangent graph, D*), etc. Ported Decision Layer (ASPEN/CASPER) to Linux Decision Layer integrated D* Path Planner

October 25, 2002 CLARAty FY02 Review- I.A.N. 37 Nesnas/JPL DECISION LAYER FUNCTIONAL LAYER Terrain Map from Laser D* Path Planner Terrain Map from Laser Path Rover Navigator Position Estimator Estimator Kalman Filter Locomotor IMU R7_Rover R8_Rover R7_Locomotor R8_Locomotor Motor R7_Motor R8_Motor Widget Board Stereo Engine Camera CameraPX610 Camera1394 FUNCTIONAL LAYER Wheel Locomotor Model Camera_Image D* Star JPL Stereo Connector CLARAty Level I Milestone

October 25, 2002 CLARAty FY02 Review- I.A.N. 38 Mars Yard Extending the Trailer Garage Door Rocky 7 going into trailer Trailer Working Desk w/ Computer The unliftable Rocky 8Main Door Sand Box (important for indoor testing) Wood to prevent sand from entering working area Start Waypoint 1 Waypoint 2 Waypoint 3 Long Range Traverse Scenario

CLARAty Development Process and Testbeds

October 25, 2002 CLARAty FY02 Review- I.A.N. 40 Repository Software Development Process AFS Backbone CMU JPL CLARAty Repository 3rd Party Releases Web UW... VxWorks ARC Rocky 8 FIDO Rocky 7 Benchtops K9 Repository ATRV Some CLARAty Statistics ~170 Modules (reusable entity) ~31 Packages (module grps) ~3 rovers ~250,000 lines of C++ code ~Java/scripts/ and models Authentication

October 25, 2002 CLARAty FY02 Review- I.A.N. 41 FIDO Benchtop Rocky 8 Benchtop Dexter Manipulators Rocky 7 Benchtop CLARAty Testbed (1/2)

October 25, 2002 CLARAty FY02 Review- I.A.N. 42 CLARAty Testbed (2/2) 5 cPCI embedded targets (x86, ppc) 2 Linux and 5 Solaris hosts/targets AFS VxWorks installations (Tornado I, II, II.2, x86, ppc) Mockups for Rocky 8, FIDO, and Rocky 7 Remotely accessible to CLARAty developers & users Web-based target status and control Small lab sandbox built for indoor testing Several remote and local users exercising testbed (MDS, CMU, ARC, JPL) Online sign up for target/rover usage

October 25, 2002 CLARAty FY02 Review- I.A.N. 43 Rocky 8 Upgrades Computational modules –CPU up to 1.2 GHz Pentium Pro w/ 256 MB (easily swappable) –FireWire 1394 Cameras (front/back) –Wireless 22 MB/sec sysLink Ethernet –I2C Master Control Board (Tracii) –Explored upgrade of widget processors to PIC18xx - product has bugs, awaiting new revision Electromechanical –Upgraded all motors to COTS motors and encoders –Rewired motor assemblies to make identical and simplify repair –New Li-ion battery system (6-7 hour operation) –Fixed steering slippage problem, upgraded camera and battery boxes.

October 25, 2002 CLARAty FY02 Review- I.A.N. 44 Rocky 7 Upgrades Computational modules –Moved from VME backplane to cPCI –CPU - PPC MHz w/ 256 MB - extended temperature –PX610 B/W Imagenation framegrabbers (x2) –New Digital and Analog I/O boards (S720 and industry pack - Acromag) –New 11 MB/s sysLink wireless ethernet Electromechanical –Fabricated new LM629 custom motion control boards (x3) –Several mechanical repairs Software –Upgraded most CLARAty drivers to support new hardware - relatively seamless transition

October 25, 2002 CLARAty FY02 Review- I.A.N. 45 Summary of Accomplishments Established a distributed inter-center software development process Provided a remotely accessible rover testbed Collaborated with MDS to define CL/MDS infusion process Developed major software infrastructure in the following packages: –Vision, Navigation, Estimation, Science, Locomotion, Motion Control, I/O, Simulation and FIDO adaptation Upgraded Rocky 7 to MSL-like hardware (PPC) Upgraded Rocky 8 to FireWire cameras Integrated Decision Layer and path planning Completed and submitted ITAR document

October 25, 2002 CLARAty FY02 Review- I.A.N. 46 FY03 Plans

October 25, 2002 CLARAty FY02 Review- I.A.N. 47 Publications & Website T. Estlin, F. Fisher, D. Gaines, C. Chouinard, S. Schaffer, and I. Nesnas, “Continuous Planning and Execution for an Autonomous Rover,” To appear in the Proceedings of the Third International Workshop on Planning and Scheduling for Space, Houston, TX, Oct 2002 I.A.D. Nesnas, “CLARAty - An Architecture for Reusable Robotic Software,” CSMISS IT Spotlight Series, June 19, 2002 I.A.D. Nesnas, R. Volpe, T. Estlin, H. Das, R. Petras D. Mutz, "Toward Developing Reusable Software Components for Robotic Applications" Proceedings of the International Conference on Intelligent Robots and Systems (IROS), Maui Hawaii, Oct Nov pdf (8 pages, 2MB) T. Estlin, R. Volpe, I.A.D. Nesnas, D. Mutz, F. Fisher, B. Engelhardt, S. Chien, "Decision-Making in a Robotic Architecture for Autonomy." Proceedings of 6 th International Symposium on Artificial Intelligence, Robotics, and Automation in Space (i-SAIRAS), Montreal Canada, June pdf (8 pages, 72KB) R. Volpe, I.A.D. Nesnas, T. Estlin, D. Mutz, R. Petras, H. Das, "The CLARAty Architecture for Robotic Autonomy." Proceedings of the 2001 IEEE Aerospace Conference, Big Sky Montana, March pdf (12 pages, 470 KB) R. Volpe, I.A.D. Nesnas, T. Estlin, D. Mutz, R. Petras, H. Das, "CLARAty: Coupled Layer Architecture for Robotic Autonomy." JPL Technical Report D-19975, Dec pdf (116 pages, 904 KB)

Thank you for your Attention