1. AUV Workbench and Autonomous Vehicle Command Language (AVCL) Progress and plans Don Brutzman and Duane Davis Naval Postgraduate School 18 October 2011

2. AUV Workbench poster

3. AUTONOMOUS UNMANNED VEHICLE (AUV) WORKBENCH Planning, generating and running robot missions

4. 4 Our 3 R’s: rehearsal, reality, replay Same needs and capabilities for each: mission, visualization, data support, etc. AUV workbench supports each ongoing work, starting to mainstream 15 years of accumulated effort integrating great variety of successful work new work projects occurring regularly Collaboration is welcome

5. Autonomous Unmanned Vehicle (AUV) Workbench builds missions, tracks Models air, surface, underwater, ground robots Currently work: remote operated vehicle (ROV) Open-source Java, XML, etc. with online archives Autonomous Vehicle Command Language (AVCL): 3-level XML-based language for robot telemetry, mission orders, mission planning Multiple converters written Tools for doing work! Mission displays, GIS, X3D graphics visualization, telemetry plots

6. 6 Rehearsal Prepare missions, either manually or automatically via other software tools Test robot software’s ability to perform commands Test again with physics “in the loop” Hydrodynamics and control are critical, difficult Sonar, environmental modeling Repeat until robust, with cautious respect “Simulation is doomed to success” – G. Bekey

7. 7 Reality: real-time mission support Monitor mission progress Task-level control using same mission vocabulary Visualize and supervise operations caveat, again: work in progress Integrate acoustic and RF communications Chat for distributed collaboration among participants, both human and robotic

8. 8 Record mission metadata for archives Support operator keeping detailed notes, kept in context when conducting mission Prompt for full details as appropriate Archive notes for later review and followup Future work Automatic tests to confirm configuration, control Automate pre-underway checklists

9. 9 Replay: post-mission support Automatic archiving of mission to server Being built into workbench – simplify user tasks Integration and compression of all relevant data into single compressed XML file Metadata for mission Many pieces: ordered mission, commands, telemetry, coefficients, contacts, etc. etc. Autonomous Vehicle Control Language (AVCL) is Ph.D. work by CDR Duane Davis

10. The Extended Rational Behavior Model Features Augments Existing Vehicle Control Software Translation to Vehicle- Specific Data Format at Lowest Level AVCL Mission Construct Reliant Minimal Modification to Existing Vehicle Controller Communications Model Pull—Higher Levels Query Lower Levels Push—Higher Levels Command Lower Levels Push—Lower Levels Provide Unrequested Information Strategic Level Tactical Level Execution Level (existing vehicle controller) Declarative AVCL Mission High-Level World Model Mid-Level World Model State and Sensor Data Translator Task-Level Behavior Script Vehicle-Specific Command Status Query Command Status Script Status Script Control Agenda Control Planner Task-Level Behavior Goal Command Flow Implied Data Visibility Optional Communications Extended Rational Behavior Model Controller

11. AVCL track, planned and simulated

12. AVCL planned mission track into KML

13. AVCL simulation track to KML

14. Extensible 3D (X3D) Graphics ISO Standard, developed by working-group members of non-profit Web3D Consortium Broad range of 3D graphics, multiple codebases Patterned after HTML and Web Architecture Evolved Virtual Reality Modeling Language (VRML) XML, ClassicVRML, compressed binary encodings Archival publishing of 3D models Geometry, interaction, navigation, DIS networking Many many resources, exemplars, archives, etc.

15. AUV WORKBENCH WORK IN PROGRESS Slow but steady progress

16. AUV Workbench efforts in progress AUV Workbench Mission metadata and mission reports Restructured threading for performance, debugging Testing AVCL support: AllCommandsUuv.xml etc. Joystick input: ROV or human-substitute control Xj3D visualization improvements Rendering support for 64-bit architectures Group merging multiple builds into common trunk Nightly builds and releases

17. AllCommandsUuv.xml List and test every AVCL command Simple testing of workbench software Confirms interface and validation Not necessarily a runnable mission Considering adding “dead reckoning” to 2D plot Mission unit tests for comprehensive coverage likely to follow Also for other robotics platforms AllCommandsUsv.xml AllCommandsUav.xml AllCommandsUgv.xml

20. Joystick input Motivation Rehearse ROV missions, especially for high-school student experiments and competitions Provide alternate control input when rehearsing complex UUV missions Make AUV Workbench suitable for classroom or poolside use Software Java jinput project handles different devices and operating systems satisfactorily

21. Thoughts on autonomous ethics Everything old is new again Largely unexplored or unconsidered territory? Significant history of experience: mine warfare? Assume success… then what? Even if perfectly executable, proper robot logic is not useful unless it is a directly extension of warfighter logic Rules of Engagement, Concepts of Operations, Doctrine, Tactics, etc. must be expressible in equivalent terms in order to be effective, usable

22. Multi-layer robot architectures 3 common layers defined in Rational Behavior Model (RBM) robot architecture Strategic: planning, goal evaluation, strategy Tactical: navigation, operations, mission conduct Execution: low-level control of systems Autonomous Vehicle Command Language (AVCL) mappable to many robot dialects Possible fourth layer: Rules of Engagement? provide ethical constraints on robot strategies

23. AUTONOMOUS VEHICLE COMMAND LANGUAGE (AVCL) XML vocabulary for robot mission specification, conduct and replay

24. Problem: Mission Incompatibility for Dissimilar Autonomous Vehicles (AVs) Vehicle-specific data formats and mission planning systems preclude effective coordination in multi-vehicle systems and hinder the design of such systems. To date, the preponderance of multi-vehicle research has assumed inherent compatibility: homogeneous vehicle systems or explicitly programmed compatibility.

25. Proposed Solution: A Common AV Data Model Common Data Format: (e.g., AVCL): Mission Specification (tasking) Communications Mission Results Automated Conversions AVCL to Vehicle-Specific Vehicle-Specific to AVCL Ultimate Goals Facilitate interoperability between dissimilar vehicles (including legacy) Support pre-, mid-, and post-mission data requirements Provide other vehicles and human operators intuitive and efficient data access AUV 2 ASV 2 AAV 2AAV 1 Support ASV 1 AUV 1 AVCL AAV 1 Support AUV 2 ASV 2 AAV 2 AUV 1 ASV 1

26. AVCL Description Autonomous Vehicle Control Language (AVCL) is a command and control language for autonomous unmanned vehicles, enabling common XML-based representations for mission scripts, agenda plans and post- mission recorded telemetry. Operators can utilize a single archivable and validatable format for robot tasking and results that is directly convertible to and from a wide variety of different robot command languages.

27. AVCL Integration in a Multi-Layer AV Control Architecture Multi-Layer AV Control Architectures Hierarchical Hybrid Relationship to AVCL Declarative Mission Scripted Mission Rational Behavior Model (RBM) Three-Level Hybrid Architecture Strategic Level: Ship CO Tactical Level: Watch Officers Execution Level: Watch Standers RBM vs AVCL: Strategic Level: Declarative Mission Tactical Level: Behavior Script Execution Level: Individual Commands

28. AVCL and Related Technologies Exemplar Data Model Definition: AVCL Task-Level Behavior Set Determination XML Schema Design Correlated information: Mission Specification (goals, constraints, tasking) Mission Results (telemetry) Communications (message set) Data Conversions AVCL translation to Vehicle-Specific languages Vehicle-Specific language translations to AVCL AVCL Declarative output as AVCL Task-Level commands AVCL Task-Level status responses to AVCL Declarative Data Model Use to Facilitate AV Control Integration into a Multi-Layer Architecture

29. Scripted Mission Specification Task-Level Behaviors Minimum Requirements to Capture Arbitrary Tasking Behavior Activation and Termination Criteria Scripting Atomic Task-Level Behaviors Sequential Execution Potential Parallelism MakeSpeed MakeDepth Waypoint MakeAltitude Waypoint ObtainGPS SetPosition Waypoint Quit

30. Declarative Goal-Based Mission Specification 14 Predefined Goal Types Finite State Machine (FSM) States Represent Individual Goals Transitions Executed upon Goal Success or Failure Mission Start Rendezvous with UUV-2 in Area C Search Area A Search Area B Sample Environment in Area A Mission Complete SucceedFail Succeed Fail Succeed or Fail

31. AVCL TRANSLATIONS AND PARSER PRODUCTION Theory and Practice

32. External C2 Systems AVCL Mission Goals and Constraints Task Level Commands Vehicle-Specific Languages Planner Rules & Templates XSLT and Vehicle- Specific XML C2IEDM Business Objects AVCL Translations Overview AVCL to Vehicle-Specific Data XSLT (text) Vehicle-Specific XML (binary) Vehicle-Specific Data to AVCL Context Free Grammars (text) Vehicle-Specific XML (binary) Artificial Intelligence Planning and Search Case-Based Reasoning Bayesian Reasoning Business Objects Command and Control Information Exchange Data Model (C2IEDM) AVCL Declarative Tasking CFGs or Vehicle- Specific XML

33. AVCL to Text—Direct generation with XSLT Exemplar Translations AVCL to Phoenix UUV AVCL to ARIES UUV AVCL to Seahorse UUV AVCL to REMUS UUV XSLT Stylesheet Vehicle- Specific Script AVCL Script Translation from AVCL to Text-Based Vehicle-Specific Formats

34. A Mutable Variable Pattern for XSLT Why? Inability to Update Variable Values in XSLT while looping (immutable properties) Requirement to Maintain Behavior Parameters for Parallel Execution How? Use Template Parameters Explicitly Controlled Iteration begin XSLT processing variable B = sequential list of task-level behaviors apply template for B 1 with default parameters d 1 to d n end XSLT processing begin template for task-level behavior B i with parameters p 1 to p n for k = 1 to n variable v k if B i updates p k v k = new_p k else v k = p k generate required output for B i apply template for B i+1 with parameters v 1 to v n end template

35. AVCL to Binary Generate vehicle-specific XML with XSLT Serialize to binary Binary to AVCL Read from binary to vehicle-specific XML Generate AVCL with XSLT Exemplar Implementation AVCL to Joint Architecture for Unmanned Systems (JAUS) Messages JAUS Messages to AVCL Vehicle- Specific Binary Custom Serializer AVCL Data XSLT Stylesheet Translation from AVCL to Binary Vehicle-Specific Formats and Vice Versa Vehicle- Specific XML AVCL Script XSLT Stylesheet Vehicle- Specific Binary Custom Reader Vehicle- Specific XML

36. Translating Vehicle-Specific Text Formats to AVCL Parse as a Context Free Language (CFL) Chomsky Normal Form (CNF) Context Free Grammar (CFG) Cocke-Younger-Kasami (CYK) Parsing Algorithm Yields a Binary Parse Tree Translate Parse Tree to AVCL Depth First Traversal Template-Based Translation of Individual Parse Nodes CFG Serves the Same Role as an XML Schema Parser/Translator Serves the Same Role as an XSLT Stylesheet Example Chomsky Normal Form Rules: Mission -> LaunchCmd + MissionMiddle Mission -> LaunchCmd + MissionEnd MissionMiddle -> WaypointCmd + MissionMdl MissionMiddle -> SurfaceCmd + MissionMdl MissionMiddle -> WaypointCmd + MissionEnd MissionMiddle -> SurfaceCmd + MissionEnd MissionEnd -> WaypointCmd + RendezvousCmd MissionEnd -> SurfaceCmd + RendezvousCmd Example (Partial) Parse Tree Mission LaunchCmd MissionMdl WaypointCmd MissionEnd RendezvousCmdSurfaceCmd

37. Translation of Declarative Missions to Task-Level Behavior Scripts Inter-Area Transit Best-First (A*) Search Goal Accomplishment Planning Trivial Goal Types MonitorTransmissions Reposition Others Require Area Coverage (Search) Decision-Tree Plan-Type Selection Parameterized Preplanned Patterns for Regularly Shaped Operating Areas Planner-Generated (A*, Hill- Climbing, Iterative Improvement) for Irregularly Shaped Areas

38. JAUS-XML possibilities? XML Binary Programming Object AVCL XSLT JAUS XML Schema Serializer Reader JAXB

39. TODO wish list for AVCL v2.1 AVCL schema needs to be simplified Remove vehicle-specific duplicate definitions Possibly streamline inner abstract types Blocker: all changes reflected in JAXB API, so XML changes provoke corresponding AUVW changes Multi-vehicle message passing, coordination Graduate improvements using MetaCommand Joystick Publish JAXB library as independent API

40. References AUV Workbench https://savage.nps.edu/AuvWorkbench AVCL page (with schema, DTD, documentation) https://savage.nps.edu/Savage/AuvWorkbench/AVCL /AVCL.html https://savage.nps.edu/Savage/AuvWorkbench/AVCL /AVCL.html Duane Davis Dissertation Design, Implementation and Testing of a Common Data Model Supporting Autonomous Vehicle Compatibility and Interoperability Design, Implementation and Testing of a Common Data Model Supporting Autonomous Vehicle Compatibility and Interoperability

41. Collaboration is welcome Naval Postgraduate School (NPS) is.mil,.edu university driven by research and education Happy to offer our resources for your use Happy to write proposals together to fund work Coming soon: 4 online courses as distance- learning certificate X3D graphics I and II, Simulation Networking, Networked Virtual Environments (NVE) Please tell us what you think. Thanks!

42. Don Brutzman, Ph.D. brutzman@nps.navy.mil brutzman@nps.navy.smil.mil http://faculty.nps.edu/brutzman Code USW/Br, Naval Postgraduate School Monterey California 93943-5000 USA 1.831.656.2149 work 1.831.402.4809 cell Contact

43. Duane Davis, CDR USN, Ph.D. dtdavi1@nps.edu Code CS/Dv, Naval Postgraduate School Monterey California 93943-5000 USA 1.831.656.7980 work Contact