1. Framework for Performance Metric Development draft-morton-perf-metrics-framework-01.txt Alan Clark alan@telchemy.com IETF 70 PMOL WG

2. Framework draft Background and motivation behind this draft and PMOL Scope – “PM Entity” however Framework “could be useful” to other WG’s Metric Development –Guidelines, criteria for defining a metric Performance metric development process –Process for proposing, reviewing metrics

3. Metrics Development View metric from “audience” perspective Definition of “metric” –measure of an observable behavior of an application, protocol or other system Tests of “Usefulness” –Essential to understand performance and find problems? –Correlates with performance as it affects the “user” –Helps with detecting, diagnosing, locating problems

4. Metrics Development Composed metric –E.g. IPPM work on spatial and temporal composition Metric specification –Process for specifying a metric (tbd) Classes of metrics –Having classes of metrics may simplify definition process Qualifying metric (testing definition of metric) –Ambiguous?, Defined Units?, Measurement Error Defined?, Repeatable?, Implementable?, Assumptions made about measured process?

5. Metrics Development Reporting models and impact on measurement –E.g. assumptions concerning sampling, averaging, window sizes…. Dependencies – Relationship between variance in “base” metrics and variance in derived or composed metrics

6. Performance Metric Development Process New proposals –Prepared as I-D’s with metric definitions that conform to previous slides –SHOULD’s be vetted by WG with domain expertize Include assessment of relevance/ interaction with other standards groups (both other WG’s and other standards organizations) Specify intended audience for metric Proposal approval –Process TBD – IETF/ IESG/ AD’s/ Relevant WGs,/ PM Entity…… Interaction with other WGs –Partnership with WG with domain expertize Standards Track performance metrics –Steer drafts through process

7. Comments, Questions, Next Steps? Potential additional definitions? –Index? –Intermediate? Other comments?