The Federal Aviation Administration Presented By: Gary Romero ATS Evaluation Programs The Federal Aviation Administration Presented By: Gary Romero

ATS Evaluation Programs Why have an evaluation? What challenges existed? What were the benefits? How is an evaluation conducted?

An Identified Need Prior to 1988, evaluations were accomplished using QA staffs within regional offices. However, this practice presented several challenges to effective evaluations.

Challenges The FAA needed to STANDARDIZE operating practices from facility to facility and region to region. FAA upper management needed NONPARTISAN, UNBIASED feedback on the performance of all air traffic field facilities. The FAA needed a way to identify air traffic field facility COMPLIANCE with national policies.

Air Traffic Evaluations was created and established in 1988

Results and Benefits Since 1988, Air Traffic Evaluations has reported individual facility compliance with national standards. Since 1999, the evaluation findings have been archived in a database that will provide future managers with important historical data to use when addressing future issues. Air Traffic Evaluations has maintained its identity as a separate nonpartisan organization, in order to continue to provide unfiltered information from the field to the upper levels of FAA management.

Results and Benefits The Sharing of Ideas: Air Traffic Evaluations has helped the FAA share good ideas, first through word-of-mouth as evaluators traveled from facility to facility and talked about useful programs and effective practices. Currently: Information is also shared electronically because each facility has access to the “National Database” where evaluation findings throughout the United States are maintained.

How is an Evaluation Conducted?

ATS Evaluation Program ATS Evaluation Process Responses Data Analysis

ATS Evaluation Process Evaluation Cycle Standardized checklist Operational monitoring Procedures review Personnel interviews Customer feedback Formal report presentation

Standardized Checklist Each facility measured against the same standard Checklist derived from ATC regulations Checklist are option specific Itemized categorized as operational or operational support Checklist is extensive—more than 200 items

Evaluation Cycle Facility changes in management and traffic levels necessitate evaluations at regular intervals

Operational Monitoring Evaluators plug-in with controllers Typical monitor times range from 30-150 hours, depending on facility size Adherence to procedures Safe and efficient operating practices Observe operational oversight Discuss observations at evaluation team meeting to determine trends

Procedures Review Are procedures in compliance with ATC regulations? Do procedures promote safe and efficient movement of aircraft? Are procedures current? Are the procedures being used? Do controllers understand procedures?

Personnel Interviews Can highlight problem areas in a facility Can be validated through research Can be an indicator of employees’ commitment to the ATC mission

Customer Feedback Contact air carrier, military and general aviation users Contact nearby ATC facilities Solicit feedback on quality of service

Formal Report Presentation Verbally present findings to management and controllers Encourage questions to ensure problems are fully understood Findings are documented in a written report

Responses Required for all problems identified in report Ensure action is taken to correct problems Corrective Action Follow-up Management Control Problem correction mandated within a predefined time limit

Data Analysis Data collected and retained for every evaluation Trend analysis conducted Preventative measures developed based on trends