1. 1 ATS Evaluation Programs The Federal Aviation Administration Presented By: Gary Romero

2. 2 ATS Evaluation Programs n Why have an evaluation? n What challenges existed? n What were the benefits? n How is an evaluation conducted?

3. 3 An Identified Need Prior to 1988, evaluations were accomplished using QA staffs within regional offices. However, this practice presented several challenges to effective evaluations.

4. 4 Challenges n The FAA needed to STANDARDIZE operating practices from facility to facility and region to region. n FAA upper management needed NONPARTISAN, UNBIASED feedback on the performance of all air traffic field facilities. n The FAA needed a way to identify air traffic field facility COMPLIANCE with national policies.

5. 5 Air Traffic Evaluations was created and established in 1988

6. 6 Results and Benefits n Since 1988, Air Traffic Evaluations has reported individual facility compliance with national standards. n Since 1999, the evaluation findings have been archived in a database that will provide future managers with important historical data to use when addressing future issues. n Air Traffic Evaluations has maintained its identity as a separate nonpartisan organization, in order to continue to provide unfiltered information from the field to the upper levels of FAA management.

7. 7 Results and Benefits n The Sharing of Ideas: Air Traffic Evaluations has helped the FAA share good ideas, first through word-of-mouth as evaluators traveled from facility to facility and talked about useful programs and effective practices. n Currently: Information is also shared electronically because each facility has access to theNational Database where evaluation findings throughout the United States are maintained.

8. 8 How is an Evaluation Conducted?

9. 9 ATS Evaluation Program n ATS Evaluation Process n Responses n Data Analysis

10. 10 ATS Evaluation Process n Evaluation Cycle n Standardized checklist n Operational monitoring n Procedures review n Personnel interviews n Customer feedback n Formal report presentation

11. 11 Standardized Checklist n Each facility measured against the same standard n Checklist derived from ATC regulations n Checklist are option specific n Itemized categorized as operational or operational support n Checklist is extensivemore than 200 items

12. 12 Evaluation Cycle n Facility changes in management and traffic levels necessitate evaluations at regular intervals

13. 13 Operational Monitoring n Evaluators plug-in with controllers n Typical monitor times range from 30-150 hours, depending on facility size n Adherence to procedures n Safe and efficient operating practices n Observe operational oversight n Discuss observations at evaluation team meeting to determine trends

14. 14 Procedures Review n Are procedures in compliance with ATC regulations? n Do procedures promote safe and efficient movement of aircraft? n Are procedures current? n Are the procedures being used? n Do controllers understand procedures?

15. 15 Personnel Interviews n Can highlight problem areas in a facility n Can be validated through research n Can be an indicator of employees commitment to the ATC mission

16. 16 Customer Feedback n Contact air carrier, military and general aviation users n Contact nearby ATC facilities n Solicit feedback on quality of service

17. 17 Formal Report Presentation n Verbally present findings to management and controllers n Encourage questions to ensure problems are fully understood n Findings are documented in a written report

18. 18 Responses n Required for all problems identified in report n Ensure action is taken to correct problems –Corrective Action –Follow-up –Management Control n Problem correction mandated within a predefined time limit

19. 19 Data Analysis n Data collected and retained for every evaluation n Trend analysis conducted n Preventative measures developed based on trends

20. 20