1. (Long-Term) Reporting Metrics: Different Points of View
Al Morton
Gomathi Ramachandran
Ganga Maguluri
March 2010
draft-ietf-ippm-reporting-metrics-01

2. Different Points of View (POV): 2 key ones
When designing IP measurements and reporting results, MUST know the Audience to be relevant
Key question: “How will the results be used?”
Network
Characterization:
Monitoring (QA)
Trouble-shooting
Modeling
SLA (or verification)
Application Performance
Estimation:
Metrics Facilitate process
Transfer-dependent aspects
Modify App Design
How can I _______ my network?
What happened to my stream?
SRC
DST

3. Versions 06 and 07 Status Last Comments from Steve Konish (2008)
Section 4 reorganized
New sub-sections to help with this.
Clarified the meaning of processing "forks"
Section 6 (Focus on Reporting) now:
Summarizes results for each metric, loss, delay delay var.
Discusses Long-Term Reporting
Advantages of Short term collection to support Long term reporting

4. Summary of Recommendations so far:
Set a LONG Loss threshold
Distinguish between Long Finite Delay and Loss
Avoid truncated distributions
Delay of Lost Packets is UNDEFINED
Maintain orthogonality – avoid double-counting defects
Use conditional distributions and compute statistics
Report BOTH Loss and Delay
Report BOTH the Sample Mean and Median.
Comparison of the Mean and Median is informative
Means may be combined over time and space (when applicable)
Means come with a weighting function for each sample if needed, the sample Size, and Loss simply reduces the sample size
Means are more Robust to a single wonky measurement when the sample size is Large
Move the Industry Away from “Average Jitter”
Use the 99.9%-ile minus minimum PDV
Portray this as a Delay Variation “Pseudo-Range”

5. New Material on Capacity
Identify two categories of Capacity-related Measurements
Raw Capacity, metrics in RFC 5136
Restricted Capacity, as in BTC Framework RFC 3148
All new material is on Raw Capacity at present
Look at the options available in terms of parameters for capacity metrics and methods of measurement
What choices seem best for Network Char? For Application Design?
Type-P, for all metrics
Capacity - Theoretical Max differs from the metric
Utilization – Time scale of averaging conceals vari.
Available Capacity – Many methods of Estimation
Variability, for all metrics …

6. New Material on Capacity
Variability, for all metrics, continued …
Two Discussion Questions:
What ways can Utilization be measured and summarized to describe the potential variability in a useful way?
How can the variability in Available Capacity estimates be reported, so that the confidence in the results is also conveyed?

7. What’s Next? Finish the WG draft on Short-term reporting?
Nothing in this draft precludes people from assigning infinite delay to lost packets – this may serve some 3rd POV
Al thinks this point resolves much of the controversy about what this draft recommends!
Need Additional IPPM people to
Volunteer to read the draft (some already have)
Suggest section reordering, if still wanted?
Discuss the Capacity related Questions now, in the halls, and on the list.