Development and Validation of Simulation Metrics FAA-EUROCONTROL Technical Interchange Meeting Performance Metrics in Air Traffic Management Toulouse, France May 2002 Amanda Stott Patricia Massimini

2 En Route Congestion Enroute Intersections At or above FL270

Terminal Area Delays Annual

4 Airspace Redesign Approach Define New Airspace Configuration Map Out Sectors and Routes Analyze Alternative Airspace Configuration Identify and Characterize Problem Areas Test with Controllers

Tools for Airspace Redesign IDATIntersect Density Analysis Toolset CRCTCollaborative Routing Coordination Tool DPATDetailed Policy Assessment Tool EACMEnhanced Airfield Capacity Model NIRSNoise Integrated Routing System MapInfo NASPACNational Airspace System Performance Analysis Capability SDATSector Design and Analysis Tool TAAMTotal Airspace and Airport Modeller TRACSTraffic Reroute and Analysis Capability System TARGETSTerminal Area Route Generation, Evaluation and Traffic Simulation SIMMOD RSATSRecombinant Sector Analysis Tool Suite CAASD ArcViewESRI CAASD FAA / Metron MapInfo Corp. CAASD FAA Preston CAASD ATAC CAASD RAMS PlusReorganised ATC Mathematical SimulatorISA Software INMIntegrated Noise ModelFAA Simulation Tools for Airspace Redesign

Throughput –Peak and average runway and airport usage Metrics from Simulation

Throughput –Peak and average runway and airport usage Delays –Arrival and departure, ground and enroute Metrics from Simulation Cumulative

Throughput –Peak and average runway and airport usage Delays –Arrival and departure, ground and enroute User impacts –Miles traveled Metrics from Simulation

Throughput –Peak and average runway and airport usage Delays –Arrival and departure, ground and enroute User impacts –Miles traveled Workload –Sector loading Metrics from Simulation

Throughput –Peak and average runway and airport usage Delays –Arrival and departure, ground and enroute User impacts –Miles traveled Workload –Sector loading –Conflicts Metrics from Simulation

Throughput –Peak and average runway and airport usage Delays –Arrival and departure, ground and enroute User impacts –Miles traveled Workload –Sector loading –Conflicts –Visualization Metrics from Simulation Others as determined by study team –Communications loading –Point outs Throughput –Peak and average runway and airport usage Delays –Arrival and departure, ground and enroute User impacts –Miles traveled Workload –Sector loading –Conflicts –Visualization

12 FAA Operational Data Traffic –Current traffic: OAG, ETMS, SAR, ARTS –Future traffic: airport authority, major carriers, ATA, FAA TAF, EIS, future traffic models Operations –OPSNET, ASPM Delays, taxi-out and taxi-in times, throughput, interarrival and interdeparture times –ASPM, ASQP Safety –Operational errors / operational deviations Traffic flow management actions –Miles in trail –Ground stops, ground delay programs

Degree of Correspondence Between Simulation Results and Operational Data Operational errors Operational deviations Vectoring Traffic flow management actions Operational Data Arrival delay, taxi out… Times for every aircraft and controller action Location and time of loss of separation if no controller action Altitudes Taxi-out times Delays Interarrival times Interdeparture times Runway throughput Simulation

14 Calibration and Validation Arrival Rates Number of Annual Arrival Pushes at Specified Rate Typically, will calibrate baseline operations –Runway arrival and departure rates

15 Calibration and Validation Typically, will calibrate baseline operations –Runway arrival and departure rates –Interarrival or interdeparture times Time Between Operations (minutes) Percent of Total Traffic ARTS Simulated results

16 Calibration and Validation Typically, will calibrate baseline operations –Runway arrival and departure rates –Interarrival or interdeparture times –Departure altitudes FT FT FT FT Typically, will calibrate baseline operations –Runway arrival and departure rates –Interarrival or interdeparture times –Departure altitudes –Taxi-out or taxi-in times –Delays Validation –Visual with controllers –Analysis of summary statistics

Example: Detroit Terminal and Cleveland En Route Airspace

CVG MSP ORD MDW STL PIT CLE DFW MEM ATL CLT BWI IAD DCA PHL EWR LGA JFK BOS DTW

Issues at ZOB and DTW Use of new runway for triple arrivals 22L 22R 21R 21L

Issues at ZOB and DTW Use of new runway for triple arrivals New arrival and departure routes and relief of MIT

DTW Issues at ZOB and DTW Use of new runway for triple arrivals New arrival and departure routes and relief of MIT Relief of congested enroute airspace Use of new runway for triple arrivals New arrival and departure routes and relief of MIT Relief of congested enroute airspace Impact on ground operations

Traffic Schedule 22 Departures Arrivals

Simulated Arrival Throughput 23 Triple Arrivals Dual Arrivals

24 $24 million per year with PRM and new procedures Effect of Triple Arrivals on Delay Applies only to IMC, MIT enforced 16% 49%

South restrictions (15-20 MIT) East restrictions (15-20 MIT) No restrictions Miles in Trail Restrictions on DTW Departures

26 Relieving MIT Departure Restrictions Reroute flows and decrease sector complexity Add departure routes

27 $24 million per year with new routes and new sectors Effect of MIT Restrictions on Departure Delay

En Route Redesign to Support Relief of Departure Restrictions 103 reroute sets defined, more than 1300 flights per day Before After 28

Approach for Analysis of En Route Redesign SDAR--No Reroute MapInfo to select and visualize reroutes Before After 29 Rerouted trajectories + Nonrerouted SAR tracks URET and SDAT to generate metrics SDAT Viewer to visualize traffic flows EXCEL to analyze and display results Altitude restrictions + Adaptation data URET restriction builder and trajectory model Custom tool to insert rerouted flight plans

30 Results of Analysis of Redesign: Max 15-Minute Sector Counts < 80% MAT 80% MAT - 100% MAT > 125% MAT 100% MAT - 125% MAT

31 Results of Analysis of Redesign: Hourly Sector Counts < 50/hour 51-60/hour >70/hour 61-70/hour

32 Results of Analysis of Redesign: Difference in Sector Counts Before and After Reroutes No change in sector count Sector count decrease Sector count increase

33 Results of Analysis of Redesign: Individual Sector Behavior Notional results Monitor Alert Parameter After Reroute Before Reroute

Results of Analysis of Redesign: Visualization of Flows

Ground Flow Analysis to Support New Runway and Triple Arrivals 35 Identified chokepoints and developed new ground routes

36 Concept Dependencies Ability to simulate is concept dependent Simulation tools may not be able to deliver all required metrics Alt 1 Alt 2 Alt 3