1. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 1 CDM Data Quality Update A&D Meeting 18 April 2001 Mike Golibersuch (Volpe/EG&G)

2. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 2 Overview Analysis of OAG-Only Time Out Cancellations (CDM Meeting Action Item) Status of Volpe Data Quality Web Site Proposed CDM Data Quality Metrics

3. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 3 Analysis of OAG-Only Timeout Cancellations Request for this analysis arose during discussion of Daily Download at CDM Meeting. The Problem: some flights are created in ETMS from OAG data, but no other messages are received and the flight does not operate. Eventually the flight Time Out Cancels. “Check and Correct” could reduce problem by detecting incorrect OAG-scheduled flights in ETMS database. Airlines asked that Volpe quantify the problem, to provide an indicator of one type of benefit obtainable with “Check and Correct”. OAG-Only TOCs were analyzed for the 60-day period from 2/5/01 – 4/5/01

4. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 4 OAG-Only TOCs Per Week By Airline (2/5/01 – 4/5/01) Strike – 3/27 ACA 3xxx  CDN (removed rule -- 4/11) USA 49xx  PDC (added rule -- 4/11)

5. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 5 Data Quality Web Site – OAG-TOC Report

6. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 6 Frequency Distribution of OAG-TOC Flights

7. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 7 Distribution of OAG-Only TOCs

8. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 8 Conclusions and Implications Problems with OAG data are episodic. Continuous monitoring is needed. “Check and Correct” capability could improve things, but benefit may be moderate. Other potential benefits of “Check and Correct” are less easily quantified.

9. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 9 Volpe Data Quality Web Site Provides access to detailed message history by flight. Two products available now to help find data problems: OAG Time-outs: highlights OAG schedule errors. GDP Time-outs: daily report of any flight in a GDP that time-out cancelled. Next Steps Performance Issues and TOC Issues Are Being Addressed Data Quality Metrics Will Be Added Access to the Site Web Site Address is 152.122.152.33:8000 Those who want access must provide client IP address to Volpe Volpe will provide User Name and Password Each Airline May View Summary Data for All Other Airlines Detailed Flight Data Only for Its Own Flights

10. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 10 Data Quality Web Site -- GDP Time-outs

11. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 11 Data Quality Web Site -- Flight History

12. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 12 ETMS/CDM Data Quality Metrics Several candidate CDM data quality metrics were discussed at the last A&D Meeting Based on general consensus, 3 Metrics are proposed for initial use as measurements of CDM data quality Each metric will be tracked and reported on the Volpe Data Quality Web Site After the measures have been tracked for a period of time, standard benchmarks will be defined Several other indicators will be tracked for analysis purposes, but at least initially will not be treated as formal data quality measures Each airline will have access to summary data (averages, trends) for all airlines, but will have access to detailed data only for its own flights

13. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 13 Data Quality Metrics 1)Timeout-Cancelled Flights As percent of:  All flights planned to operate (FS, FC, or FM)  All cancelled flights 2)Airline-Cancelled Flights (FX, SI) Flights that Operate As percent of:  All flights planned to operate  Airline-cancelled flights Reinstated flights (FX followed by FC) are OK, and are not counted 3)Unscheduled Flights (no OAG data) that Operate without a CDM Message As Percent of:  All flights that operate  Unscheduled flights that operate

14. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 14 Metric #3. Unscheduled Flights that Operate without CDM Message

15. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 15 How Will the Data Be Displayed Bar Charts comparing average values by airline over a user- specified time interval Trend Line showing weekly value over 4-month timespan for a selected airline Airline may request a list of its flights that match the criteria during a specified time interval Airline may select a flight from the list to display flight details and message history

16. Volpe Center AUTOMATION APPLICATIONS DIVISION DTS - 56 4/18/01 16 Other Indicators 4)Scheduled flights that operate without a CDM message As Percent of all scheduled flight that operate Useful for detecting problems with daily download 5)Flights that depart at least 15, 30, 45, or 60 minutes after EDT * As Percent of: –All flights that operate –All delayed flights (I.e., actual departure after OAG schedule) 6)Controlled flights that depart at least 15, 30,45, or 60 minutes after EDT * As Percent of controlled flights that operate  For measurement purposes, EDT is determined as follows: Use ETMS EDT 10 minutes prior to actual departure. I.e., for measuring prediction accuracy, disregard last minute updates From this EDT, remove any Time Out Delay that was automatically added by ETMS