Presentation on theme: "Dr. Professor Francisco Saez & Professor Fernando Gomez"— Presentation transcript:
1 Are homogeneous performance metrics for the whole ATM system life cycle available? Dr. Professor Francisco Saez &Professor Fernando GomezDepartment of Aeronautical Infrastructure, Air and Space Systems and Airports. Polythecnic University of MadridJohn N. Polky, Ph. D.Boeing Research & Technology Center, S.L.
2 ATM community objective: Objective of the workIllustrate the existent gap between high level performance indicators used normally for setting concept performance targets (ATM system capacity increase,etc... ) and low level metrics and indicators.Identify existing and recognised links.Identify those areas where additional R&D effort is still required for quantifying and modelling the various links and contribution between metrics.ATM community objective:To have available a measurement framework (an homogenous and coherent set of metrics) for the whole concept life cycle.
3 Context the new concepts definition process CCDTI suitabilitynumber of resolutionadvisories, etc....Which are the performances that really characterise the ATM System?TOO CONSTRAINEDBY THE IMPLEMENTATION!post implementation evaluation?targets settingTOOHIGH LEVEL!ATM 2000+SAFETY,CAPACITY,ECONOMICS....PRCDELAY, ACCESSFLEXIBILTY.validationTOOLOW LEVEL!Controller workloadnumber ofconflicts, etc....
4 Context the performance measurement framework? The contribution of low level indicators to the high level indicators should be quantified and unique mathematics relationships should be established.That work is fundamental for completing the overall validation frameworkWhich is the relationship between low level measurements and high levels performances?Which are the performances that really characterise the ATM System?R&D is required for Identify and develop the mayor gaps between High and Low level indicators
5 ATM system performance measurement High level performance areas vs. Local performance elements (I)ATM SYSTEMGLOBAL indicator?Local IndicatorOutcomesResults forcustomersOutputsProducts and servicesProcessesActivities to achieve outputsInputsResourcesATMsub systemGLOBALINDICATORSLOCALINDICATORSModern approach to theATM system performance measurementFocus on value of service provided to the userExtension beyond traditional measures of capacity and delay
6 Complex and not characterise relationships between: High level performance areas vs. Local performance elements (II)Complex and not characterise relationships between:High level/Globalperformance objectives(areas, performance...)Specific or Local performance indicators (factors, metrics,...)+-Specific or Local performance indicators are related to system components ( either airborne, ground systems, pilots, controllers, procedures,etc...High level/Globalperformance objectives are related to the Overall ATM systems performancesContributes or not to achieve the target performance objectives
7 High level performance areas vs. Local performance elements (III) Examples:High level/Globalperformance objectives(areas, performance...)Specific or Local performance indicators (factors, metrics,...)Factor that contributes to delay?:Controller workload:PRC/PRU:Performance area KPA DelayLocal indicator :Percentage of time spent of the frequencyNumber of aircraft handled.Global Indicator KPITotal minutes of delay/Total number of flightsCapacity index (increase in movements for a given ATFM delay)
8 - + + - -5% +30% Performance indicators framework elements Pilot’s workloadEquipmentController’s workload...Controller’s situational awarenessSeparations standardsConflictsDelaysStressFinal user perspectiveSafety...CapacityEfficiencyCost effectiveness-5%-++30%Pilot perspective+-ATS perspectiveOther stakeholders perspectiveFactors contributing toATM System performancesLocal indicatorsPerspectivesATM System performancesGlobal indicators
9 PHASE 1 PHASE 2 ON GOING WORK PHASE 3 PHASE 4 Work to be done Documentation anddatabases analysisApproach definitionMethodology definitionStructure andframework definitionGaps and R&D needsidentificationSupporting databasecreationMetrics populationof the databaseFramework structurevalidity evaluationFramework rework and upgradeON GOING WORKPHASE 2PHASE 3Metrics populationof the databaseFeedback evaluationFramework rework and upgradeFramework andsupporting databasediffusionR+D actionsLinks andrelationshipsmodellingContinuous metrics assessmentContinuosFramework rework and upgradePHASE 4
10 Performance indicators framework scheme Stakeholders/participantsperspective orpoints of viewHigh level /Global ATMPerformance areasorsystemperformancesCODATORCHPRCATM2000+OCDCAF/TLocalFactors------workloadMetrics/measurements------Total minutes onthe frecuencyATSAirlinesAirpotsFinal users----HumanfactorsAtomatizacionPerformancesareascategorisation------DelayMetrics/measurements------Total minutes of delay/Total number of flightsLow level /Localindicators orsubsystemperformancesSeparation standardsATFMCNSIdentify the missing contributionsDevelop a structured framework that allows interrelationships.
11 High/global ATM system performance level Developing a structured framework that allows interrelationships. (I)High/global ATM system performance levelUPPERStakeholderview relevant to:An unique and structured framework of performances that allows or accommodate the various stakeholders views.Each ones of the performances area will be characterised by their intrinsic characteristics and by the relationships with the others .Global indicators are proposed for each one of the performance that are their intrinsic characteristicsMetric/units/measurements are compiled for each intrinsic area or global indicatorStudied sources:OCD, ATM2000+, PRC, CAFT, INTEGRA, TORCH, NASCOLATERALLDELAY:IntrinsiccharacteristicsLOWERGlobal indicatorsMetrics/measurements/units
12 ASSESSMENT OR MODELLING Developing a structured framework that allows interrelationships. (III)ASSESSMENT OR MODELLINGLocal Indicators
13 ASSESSMENT OR MODELLING Developing a structured framework that allows interrelationships. (IV)Performance aggregation processOnce this low level characterisation is done performance aggregation models should be consideredASSESSMENT OR MODELLING
14 decomposition of factors s Final productsStakeholderviewsGaps and R&DneedsidentificationMeasurementframeworkPerformanceareaLife cyclestateRelationaldatabaseSubsystemunderanalysisAgregationmodelsStructured breakdowndecomposition of factors scontribution toeach performancePerformanceroadmap
15 Evolution of the approaches towards performance metrics Contributes to CharacteriseOperationalimprovementsContributes totypify thereceived servicePRCKPAOCDATM 2000+CAF/TSafetyCapacityEfficiencyEnvironmentSafetyCapacityEfficiencyEnvironmentSecurityEconomicUniformityQualityDelayPredictabilityFlexibilityEfficiencyAccessCost of serviceSafetyDelayPredictabilityAccessFlexibilityAvailabilityEquityFlight efficiencyCost-effectivenessEnvironmentProductivityCapacityTrafficSafetyDo not reflectperspectives ofthe stakeholders.ConceptualLevelUserorientedviewNo explicitmeasurementsproposed
17 Common conceptual definitions and aims but different metrics CAF/T versus PRC approachPRCKPACAF/TIt accounts for the time in excess of the optimum time that it takes a user to complete an operationIt accounts for the abitly of the user to predict variation and build and maintain optimum flight schedulesDelayPredictabilityFlexibilityEfficiencyAccessCost of serviceDelayPredictabilityFlexibilityFlight efficiencyAccessCost-effectivenessAvailabilityEquityEnvironmentSafetyCommon conceptual definitionsand aims but different metricsIt accounts for the ability of the system to accommodate changing user needs in real time and without penaltyIt accounts for the abitly of the ATM system to allow the user to adopt the preferred flight profile (route and level)It accounts for the ability to enter the ATC system and obtain services on demandThe value for the money that the user received from the ATSMeasures the availability of critical ATM resources and servicesMeasures of an output over the cost to produce itProductivityCapacityTrafficSafetyNumber of operations using a resource in a specified period of timeMeasures equity of treatmentMeasures the conformance of air transport to environment regulationsNumber of total operation (flight passengers)Measures exposure of certain incidentsMeasures the conformance of air transport to specified safety targets
18 FUSION and STRUCTURATION Framework approach (I)European Performance Review System:CRCO,STATFORD,CFMU,CODA,EATCHIP CIP monitoringOCDCAF/TMAEVAR+DprojectsresultsFAAperformancemeasurementdatabaseATM2000+PRCAirspaceManagementhandbookFUSION and STRUCTURATIONCAPACITYSAFETYENVIRONMENTEFICIENCYDELAYFLIGHT EFFICIENCYPREDICTABILITYCOST OF THE SERVICEACCESSThirteen top ATM issuesFLEXIBILITYPRODUCTIVITYAVAILABLITYEQUITY
19 Example: Delay Performance Breakdown decomposition Identifying factor that contribuyes to the ATM system performances!Example: Delay Performance Breakdown decompositionMain assessed approaches:Incremental approachPRCapproachInitial Delay globalindicators organisationInitial delay attributes characterisationInitial delay breakdown decompositionCAF/Tapproach+Initial identificationof factor and subsystemcontributing to the total ATMsystem DelayCFMUapproach+Proposal of aggregation models.CODAapproach+Integrated delay measurement frameworkdelay intrinsic attributesdelay indicators characterisationbreakdown decompositionaggregation modelR&Dprojectsapproach++Operationassessmentapproach
20 Delay key attribute extension Initial DELAY intrinsic attributesAbstracted from PRCDelay key attributesTotal delay/Total Nº of flightsMAGNITUDSEVERITYABSORPTIONBreakdown decompositionby contributions(Phases of the flight operation, before and during the flight)Nº of delayed flights/Total Nº of flightsDELAYTotal delay /Nº of delayed flightsDELAYDELAYDELAYDELAYDELAYDELAYDelay key attribute extensionDELAYDELAYTD/Nº TFMAGNITUDSEVERITYABSORPTIONNº DF/Nº TFCausal Factor =Delay/causeDELAYTD/NºDF
21 (PRC) PRC+CODA Extension PRC+CODA+ Operational assessment views Extended DELAY breakdown decomposition: Contributions to the total delay(PRC)PRC+CODA ExtensionPRC+CODA+ Operationalassessment views
22 Initial Delay Aggregation Model DT/Nº TVMAGNITUDSEVERITYABSORPTIONNº VD/Nº TVBreakdown decompositionby contributions(Phases of the flight operation, before and during the flight)DELAYDT/Nº VDDELAYDELAYDELAYDELAYTotal delay/Total Nº of flightsNº of delayed flights/Total Nº of flights* Without repetition nor duplicationTotal delay /Nº of delayed flights
23 Mixed time references systems and origins (I) Several time reference system are used for measuring delays:There is a need of defining those references clearly and stabilised how to pass from one to another. Co-ordinates translationActual timesDifference between actual and schedule timeScheduled timesDifference between schedule and planned timePlanned timesDifference between planned and preferred timePreferred timesDifference between optimum and preferred timeOptimum timesPRC Defines :Optimum timesPreferred timesPlanned timesScheduled timesActual timesEventat TnDifference between actual and optimumDifference between actual and planned timeDelay can be calculate as difference between the various time reference as convenience
24 ? DELAY Capacity Index Areas requiring further R+D Capacity CAPACITY place for R&D actions in the delay measurement framework?DT/Nº TVMAGNITUDSEVERITYABSORPTIONNº VD/Nº TVDT/Nº VDCausal Factor =Delay/causeDELAY?CapacityIndexDelay/traffic curves as a measure of capacity variationAreas requiringfurther R+DCapacityCAPACITYSpecially attention should be pay on traffic patterns definition when relating delay and capacity.Lack of models for understanding the high level relationships between performance parameters.Extension of the breakdown decompositionDefinition of proper mechanisms that relates the various time reference systems so mechanisms for translation are available.Mapping delay metrics to the proposed structure and database population
25 Conclussions. Project object To have available a measurement framework (an homogenous and coherent set of metrics) for the whole concept life cycle.To unify sources and stakeholders views.To be corrected and completed by ATM CommunityTo be used by anyone that could be interested in.