1. Operational Considerations in the Certification of TAA Slide 1 Operational Considerations in the Certification of Technically Advanced Aircraft (TAA) Robert Barnes, Robert B Barnes Associates, Inc. (USA) Charles Adam, Federal Aviation Administration (USA)

2. Operational Considerations in the Certification of TAA Slide 2 Today’s topics The need to consider the user What we’ve seen during flight test Current policy Current design practice Importance of real user trials Our suggestions

3. Operational Considerations in the Certification of TAA Slide 3 “General aviation today is experiencing the most rapid pace of innovation since the late 1940s … … These advances offer general aviation pilots the promise of increased levels of safety and performance … … Unfortunately, these increased levels of safety have not materialized.” -- Littlefield, Glass Cockpit Flying, 2010

4. Operational Considerations in the Certification of TAA Slide 4 “Why has the promise of greater levels of safety for glass cockpit airplanes not been realized? Because, in order to realize these benefits, general aviation pilots must learn a new way of flying. Unfortunately, general aviation pilots and training providers have not yet evolved the way they train and fly to catch up with the advances in glass cockpit technology.” -- LinkedIn Aviation Training Group Discussion

5. Operational Considerations in the Certification of TAA Slide 5 As seen during flight test Design may be functionally acceptable to engineers Is the design acceptable to properly qualified pilots? Can a crew safely operate new design?

6. Operational Considerations in the Certification of TAA Slide 6 Some personal findings Engineers tend to design systems which may seem logical and intuitive to them (an engineering skill) –Sequences of steps –Multifunction control logic These same systems may be far from logical or intuitive to the typical pilot (a pilot skill) –Recognizing operation of dedicated controls

7. Operational Considerations in the Certification of TAA Slide 7 Effectively demonstrating design acceptability requires: Involvement of qualified pilots … Representing a cross-section of the target pilot population … Who are operating the new design under realistic conditions.

8. Operational Considerations in the Certification of TAA Slide 8 Human Factors in Part 23 Aircraft Certification FAA Certification Policy on Human Factors (PS-ACE100- 2001-004, Aug 2002) - Response to industry requests for guidance - Advocates early FAA involvement during development - Goal: To aid in the timely identification and resolution of human factors-related issues during the aircraft certification process - Guidance only … not a regulatory requirement “General Aviation fatalities (part 91 operations) occur at higher rates than part 121 and part 135 operations.”

9. Operational Considerations in the Certification of TAA Slide 9 Operational Considerations Pilot characteristics Operational considerations System operation procedures Training requirements

10. Operational Considerations in the Certification of TAA Slide 10 FAA Policy on Human Factors Not a regulatory requirement but FAA ACO “should attempt” to follow Increased effort being made to address design- related human error FAA rulemaking effort (25.1302) already being applied by EASA Human factors issues are open to broad interpretation –Component level –Operational level

11. Operational Considerations in the Certification of TAA Slide 11 Human Factors Design Practice Design and Operational factors affect workload There are many ways to evaluate these affects 500 pages of methods described User trials are on page 475

12. Operational Considerations in the Certification of TAA Slide 12 Some user trial advantages Simplistic and flexible approach to usability evaluation Evaluates the system based upon the potential end users’ performance Gives the designers a powerful insight into how the system will be used

13. Operational Considerations in the Certification of TAA Slide 13 Some user trial disadvantages Time consuming to conduct Large amounts of data are collected, ensuring a lengthy analysis process It may be difficult to gain access to the required personnel or end users End users may be biased towards the old system or procedure

14. Operational Considerations in the Certification of TAA Slide 14 Applying User Trials in U.S. Limited form now used periodically by FAA for avionic component evaluations Multiple Pilot System Usability Evaluation (MPSUE) FAA pilots not typical end users Does this adequately evaluate operation by the typical user?

15. Operational Considerations in the Certification of TAA Slide 15 TAA certification user trial Removing restrictions required a human factors workload evaluation Applicant chose to use Modified Pilot Subjective Evaluation (MPSE) Two typical user pilots were qualified Certification was successful but many lessons were learned by manufacturer, training provider, and regulator

16. Operational Considerations in the Certification of TAA Slide 16 Conclusions Human factors evaluation is subjective Level of pilot competence required to operate design is critical Using experienced test pilots is not enough “Is the aircraft capable of being operated safely by the manufacturer’s target customer with the type of training proposed?”

17. Operational Considerations in the Certification of TAA Slide 17 Questions/Comments? For a copy of our papers, please leave me your card