Presentation is loading. Please wait.

Presentation is loading. Please wait.

Flight Software Workshop

Similar presentations

Presentation on theme: "Flight Software Workshop"— Presentation transcript:

1 Flight Software Workshop
December 16 – Safe Testing of Autonomy in Complex, Interactive Environments (TACE) David Scheidt Robert Lutz William D’Amico Subodh Harmalkar

2 Outline Test & Evaluation Need and Challenges
Background and Definitions Current Effort under TRMC’s T&E/S&T program Path to a Testing Capability Summary

3 From Tele to Auto to Autonomous Operations
Unmanned systems usually require the operator to be a “user-in-the-loop” APL has demonstrated the autonomous operation of multiple and cooperative UXVs where the operator is a “user-on-the-loop” There have been many demonstrations of APL’s Mission Level Autonomy (MLA) under JHU/APL internal research and at Camp Roberts (CA) MLA has been shown to be “platform” and “vendor” agnostic riding above proprietary control/guidance algorithms MLA research led to insights on what would be needed for rigorous test and evaluation UXV Sensor Control Autonomous Operations Operational objectives Measure effect Measure effect Operating environmental decisions A measure of effectiveness of an autonomous system is directly related to “operational objectives” thus making the test methodology and test results applicable to both developmental and operational test environments – true joint DT/OT is needed

4 Systems with Autonomous Behaviors Are Here
AACUS: Autonomous Aerial Cargo/Utility System ACTUV: ANTI-SUBMARINE WARFARE CONTINUOUS TRAIL UNMANNED VESSEL AMAS: Autonomous Mobility Appliqué System LDUUV: Large Diameter UUV

5 Autonomy Test & Evaluation Challenge
Autonomous systems respond to unpredictable change by devising a course of action. Before we deploy such systems how can we be sure that autonomous decisions that will be produced will always: (1) achieve objectives set by human supervisors and (2) not produce unacceptable unintended consequences. Testing autonomous systems is particularly challenging since we cannot possibly test all interactions between the autonomous system and the natural world.

6 History of Autonomous Unmanned Systems at APL
JHU/APL Proprietary History of Autonomous Unmanned Systems at APL JHU/APL has been developing autonomous unmanned vehicles since 1962. 4 APG OPISR – 4 UAVs, 1 UGV, 2 USV, 1 UUV, 3 UGS Tactical Sensing 1 UAV & Multiple UGS @ Webster Predator/SSN Interoperability Demo 2 UAVs & 4 UGVs Unmanned Surface Vehicles New Horizons 1 UAV 1 UGS @ APG Air-Exjam (later EXDRONE and BMQ-147 Dragon) @ APG Autonomy S&T Vision Program 1965 1970 1990 2000 2005 2010 2015 TACE 3 UAVs, 2 UGS, 3 Mobile Users DADFS Agile UxV Autonomy 1 UUV @ Dugway & DOE Range TNT Experiments 6 UAVs, 3 UGS Small Ocean-going USVs Scheer “Beast” DARPA UUV Program NEAR burn failure Since 2002 JHU/APL has conducted dozens of autonomous vehicle flight programs and flown/launched hundreds of sorties. Currently JHU/APL has 24 active autonomy programs. In Atlantic, Pacific & Gulf of Mexico @ Camp Roberts

7 Combined Autonomous Air/Ground Missions 2004 Aberdeen Test Center
OSD NII Swarming Unmanned Vehicle Experiment – Cooperative Search, Patrol & Track (4 UGVs and 2 UAV) Combinations of autonomous air and ground vehicles (an AACUS/AMAS collaboration) are clearly possible and probably required for many military missions

8 Challenges for Autonomous System Test
Test & Evaluation (T&E) Need Build an infrastructure for SAFE/LIVE testing of autonomous unmanned vehicles (AUVs), especially for tactical UAVs that have significant airspace and platform- related range safety issues Demonstrate live, virtual, and constructive (LVC) methods for SAFE/LIVE tests LVC methods should be Test and Training Network Architecture (TENA) compliant To our knowledge there are no AUV established T&E capabilities with these features T&E Challenge Provide an infrastructure that supports safe testing when testing autonomous systems operating over the horizon or in denied environments. Provide an infrastructure that supports safe testing when autonomous systems perform unpredictable actions Provide an accurate, real-time, live-virtual-constructive environment that interacts with the system under test’s autonomy in unpredictable ways. Reliable command and control (C2) of the systems under test (SUT) and robust 2-way data links are needed to stimulate the SUT, to record behaviors, and to maintain SAFE/LIVE test control

9 Safe Testing of Autonomy in Complex, Interactive Environments (TACE) – Key Elements
Capability TACE is a 3-phase/36-month program (started in April 2013) where the open/general architecture will be at TRL6 with initial capabilities for the testing (black and white box) of autonomous unmanned vehicles (AUVs) of all types. TACE is “portable” but will rely upon a “thin client” interface to the SUT and the inclusion on the SUT of a minimal “TACE test applique.” There are 3 levels of control/override within TACE: SUT autonomy, TACE automatic experiment controls, test range safety officer control Relevance to T&E Scenarios The TACE architecture will use rigorous techniques to prevent unsafe AUV operations or actions while stimulating, measuring, monitoring, and recording the AUV’s autonomous response/performance in complex environments. TACE assurance algorithms provide mathematically rigorous, on-board, real-time, safety guarantees, while TACE dual simulation system provides a complex live-virtual constructive test environment. Cost Benefits A critical element, as always, is selecting the critical tests. TACE is not a planning tool. A recent award was made to JHU/APL under the Unmanned and Autonomous System Test (UAST) program for an autonomy planning tool, Rapid Adversarial Planning Tool (RAPT). The “cost” of conducting tests where the “Achilles heel” of the embedded autonomous behavior is unknown would involve very high risk to the loss of the SUT and the delay of system fielding. The combination of tools such as RAPT and TACE should prevent unnecessary and expensive testing

10 TACE System Architecture

11 TACE System Under Test (SUT )Payload
TACE’s payload for the Boeing ScanEagle is a repurposing of JHU/APL’s Autonomy Toolkit (ATK) software, which has been used on more than a dozen types of UAVs in Boeing, Lockheed, Aerovironment manufactured vehicles, and hardware previously developed jointly under internal research funds by Boeing and JHU/APL ATK payload with Persistent Systems Wave Relay Wireless Local Area Network Card TACE SUT payload installed in a Boeing ScanEagle payload bay

12 TACE Flight Tests at Aberdeen Test Center
Five Test Events with Multiple Sorties Were Executed during January/February 2014 APL Test Team on the tarmac at Phillips Army Airfield (PAAF) Aberdeen Test Center (ATC) Hand launch of the Procerus research AUV controlled by JHU/APL’s Autonomy Tool Kit (ATK)

13 Sample Results from TACE Testing at ATC
Note the bank angle of the plane in the lower left corner. SUT Moving NoGo Virtual Target Safe Loiter Point Fixed NoGo Area Constraint Violation - Range Safety Executive User Interface

14 Path to a Testing Capability – Phase 2 OV1
ScanEagle as the SUT at Yuma Test Center Restricted Airspace(s) Virtual JIMM Entities 1 to N ScanEagle SUT-System Under Test Watchdog Behaviors 3D Range Boundaries Platform Safety Limits Virtual Translated Air Traffic Tactical EO/IR Live Unicorn UAV Autonomous Behaviors Search, Track, Avoid Unicorn UAV Onboard Test Payload Test Range Topology Wave Relay 2400MHz UHF PNT Downlink UHF Safety Link Native UHF Link Live target track to SUT Test Director Range Safety Test Ground Station Intruder C2 Intruder GS Land Lines Defense Research & Engineering Network Connection

15 Flight Test Operations Base
Notional Playbox at a Large Test Range 1.0 Mile Geographic Constraint Boundaries 10 Miles Flight Test Operations Base 7.5 Miles 2.5 Miles 1.0 Mile 2.0 Miles 1.0 Mile

16 Proximity Violation (Virtual Entity)
Note: paths shown are not actual paths) Track Established Violation Virtual Aircraft Live Ground Entity

17 Summary for Autonomy and T&E Perspectives
TACE Phase 3 will provide some basic operational functions at TRL 6 – this is an “initial architecture” but not a “range capability” To our knowledge, there are no “active” programs to extend beyond TRL6 – highly understandable since there are no PORs with autonomous behaviors At the recent NDIA Annual T&E conference – SHIFT LEFT is coming RFPs will include CONOPS and initial TEMPs - will “autonomy” be a “shift left” – Insights from Dr. Brown, Director TRMC - cited “cyber, big data, hypersonic, and autonomy” as the T&E challenge areas for the near future The NATO Science and Technology Organization (STO) promotes and Validation and Verification of Autonomous Systems (SCI-274)

18 Acknowledgements JHU/APL TACE Team: Robert Chalmers, Robert Bamberger, Kristine Ramachandran, Dean Kleissas, Brendan John, Subodh Harmalkar, William Van Besien, Michael Biggins TACE Subcontractors to JHU/APL : Trideum: Michael O’Connor Boeing Corporation: Gabriel Santander InSitu Inc.: Jerry McWithey Government Flight Test Support: Aberdeen Test Center: John Wiley Yuma Test Center: Mary Beth Weaver and Bob Vondell Unmanned and Autonomous Systems Test Test Technology Area Executive Agent: Vernon Panei Deputy Executive Agent: Stephanie Riddle Subject Matter Experts: William Hamel, Kirk Bonnevier


Download ppt "Flight Software Workshop"

Similar presentations

Ads by Google