1. “Flight Test Is As Flight Test Does” Guide to Safe and Effective Flight Test George Cusimano Flight Test Safety Workshop 5 May 2010 1

2. Mamma always said: “Stupid is as stupid does” Forrest Gump circa 1994 Meaning – You are what you do Applies to flight test as well as life in general 2

3. Safe & Effective Flight Test Is... Desired perceptions – Culture of safety – Manage risks – Credible and unbiased – Efficient and effective – Legitimate results – Requirements validation 3

4. ... As Flight Test Does Real actions do not always reflect good intentions – Program pressure – test is an obstacle to progress Cost Schedule – “We can do more with less” attitude – Poor risk assessments Inadequate Incomplete – Poor systems knowledge 4

5. ACTIONS Guide to Positive ACTIONS Truths for Safe and Effective Flight Test – Common sense – Historical basis – Mindful of safety “Stupidity” prevention 5

6. ACTIONS Guide to Positive ACTIONS All models are wrong Believe the data until it is proven wrong Don’t be like a box of chocolates Always challenge When in doubt – Don’t Testing the wrong thing right is as costly as testing the right thing wrong If at first you don’t succeed – check your plan 6

7. All Models are Wrong! Model: Representation of reality – Bounded by assumptions and simplifications – Valuable to the development process Flight Test: Reality – Validates models – If predictions are incorrect – fix the models! 7

8. Believe the Data... Until.. Flight Test is truth... IF – Good instrumentation – Test configuration – On the correct condition – Applying the correct technique, correctly If the results are different than expectation... – Slow down – Stop... until the differences are resolved 8

9. Believe the Data (Example) A tracking algorithm was supposed to run at 100 Hz. During test the control room observed 10 Hz – Assumed to be a TM anomaly Post flight analysis revealed that 10 Hz was real. 9

10. Box of Chocolates Mamma always said: “Life is like a box of chocolates, you never know what you are going to get” Forrest Gump circa 1994 Flight test should NEVER be like a box of chocolates; we should always know what to expect! 10

11. Set Expectations Predict flight test results – For every test point – Based on models and/or analyses If you do not know what to expect, then you do not understand the system If you do not understand the system, then you may not be testing the right thing right If you are not testing the right thing right, then you should not be conducting the test. 11

12. Set Expectations (Example) Objective: Expand forward CG limit – 12” forward of existing limit Incremental approach – First flight planned @ 1” in front of baseline limit Difficulty rotating with full aft stick; nearly departed Did not understand why – decided to land... 12

13. Always Challenge What if something unexpected happens? – What will we do? – Have a plan Why did the system behave as it did? – Systems knowledge – Significance of error What next? – Do nothing – proceed – Suspend test point, but continue the mission – Stop the mission 13

14. Always Challenge (Example) Designed to test fighter thrust vectoring technology High AOA test point Pilot: “Something is not right. I show 280 KIAS and 20 alpha.” Pilot did not persist in knowing why Control room did not suspend test while they investigated Result: Lost aircraft 14

15. When in Doubt – Don’t! If test results do not meet expectation – Ponder – Wonder – Debate – Decide Listen to your instincts The obligation to stop a test resides with every member of the test team Proceeding without knowledge = disaster 15

16. When in Doubt (Example) Objective: Expand forward CG limit – 12” forward of existing limit Incremental approach – First flight planned @ 1” in front of baseline limit Difficulty rotating with full aft stick; nearly departed Did not understand why – decided to land.... & try again Repeated problem on 2 nd takeoff Post flight – CG 13” in front of existing forward limit 16

17. Testing the Wrong Thing Right; Testing the Right Thing Wrong Both are costly and inefficient How we get trapped: – Not having well defined test objectives – Not understanding the system under test 17

18. Testing Right (Example) RADAR test at maximum slant range Weather forced the test to higher altitude and closer horizontal range Four hours of data collected Result: Invalid test – Test had to be repeated 18

19. If at First You Don’t Succeed -- Check Your Plan Based on well defined requirements – Relevant test objectives – Data required to meet the objectives – Methods needed to collect that data – How the collected data will be analyzed to show compliance – Must include resources 19

20. Check Your Plan (cont.) Without a good test plan you do not know: – Why you are testing – How you will test – When you are finished testing – What to do with the collected data Plans change – Configuration control is required 20

21. Check Your Plan (Example) F-117 Night Navigation triangles with autopilot Pilot decided to hand fly one turn – Disengaged autopilot – Black hole phenomenon Almost departed the aircraft. 21

22. Configuration... Test article – Testing the right thing Test plans – Testing requirements Instrumentation – Valid data Test methods and conditions – Testing right Know What You’re Testing 22

23. Common Element Test Article Test Plan Instrument Methods & Conditions Models are Wrong Not real Limited by simulation Not part of simulation Limited by simulation Believe the Data  Set Expectations  Always Challenge  When in Doubt  23

24. Common Element Test Article Test Plan Instrument Methods & Conditions Test the Right Thing  Planning is Everything  24

25. Summary “Stupid is as stupid does” – Testing without knowledge is stupid Do NOT be like a box of chocolates – Always know what to expect 25 Know What You are Testing

26. “That’s all I have to say about that !” “That’s all I have to say about that !” Forrest Gump circa 1994 26

27. Models – Non-linear aero assuming a rigid structure – Aero with quasi-aeroelastic effects Goal: Determine stability margins Each model yields different results – Use models to bound possible margins – Assure adequate design to handle worst case Flight test to identify stability parameter All Models are Wrong! (Example) 27

28. Summary “Safe and effective flight test is as flight test does” – Flight test = truth – Have a plan – Set expectations – Always challenge – Proceeding without knowledge = disaster 28