1. Delay Variation Applicability Statement draft-morton-ippm-delay-var-as-04 December 3, 2007 Al Morton Benoit Claise

2. Page 2 Applicability Problem: “How will the results be used?” Krzanowski introduced the Delay Variation Problem at IETF-64 “How” Question asked at IETF-65, no suggestions yet RFC 3393 lists two key uses for the Delay Variation Metric Memo Considers these 3 Tasks and N Special Circumstances SRC DST Network Characterization: Inferring Queue Occupation on a Path Application Performance Estimation: Sizing of De-Jitter Buffers What about Spatial Composition ? What if circumstance s are challenging?

3. Page 3 Inter-Packet Delay Var. (selection func = previous packet) Src Dst T1 R1 T2 R2 T3 R3 T4 R4 T5 R5 IPDV(2) = (R2-R1) – (T2-T1) (5)(2)(4) IPDV(4) = (R4-R3) – (T4-T3) (3)

4. Page 4 Packet Delay Variation (selection func = minimum delay pkt in stream) Src Dst T1 R1 T2 R2 (minimum) T3 R3 T4 R4 T5 R5 PDV(3) = (R3-T3) – (R2-T2) (2)(4) PDV(4) = (R4-T4) – (R2-T2) (1,3,5)

5. Page 5 Current Outline of the Draft 1. Introduction 1.1. Background Literature in IPPM and Elsewhere 1.2. Organization of the Memo 1.1.1.1.2. 2. Purpose and Scope 2. 3. Brief Descriptions of Delay 3. Variation Uses (four) 3.2. De-jitter Buffer size3.2. 4. Formulations of IPDV & PDV 4. 5. Earlier Comparisons 5. 6. Additional Properties and Comparisons 6.1. Packet Loss 6.2. Path Changes 6.2.1. Lossless Path Change 6.2.2. Path Change with Loss 6.3. Clock Stability and Error 6.4. Spatial Composition 6.5. Reporting a Single Number 6.6. Jitter in RTCP Reports 6.7. MAPDV2 6.8. Load Balancing 6.6.1.6.2.6.2.1.6.2.2.6.3.6.4.6.5.6.6.6.7.6.8. 7. Applicability of the Delay Variation Forms and Rec 7.1. Uses 7.1.1. Queue Occupancy 7.7.1.7.1.1. 7.1.2. De-Jitter Buffer Size 7.1.3. Spatial Composition 7.1.4. Service Level Specs7.1.2.7.1.3.7.1.4. 7.2. Challenging Circumstances 7.2.1. Clock Issues 7.2.2. Frequent Path Changes 7.2.3. Frequent Loss 7.2.4. Load Balancing 7.3. Summary Table7.2.7.2.1.7.2.2.7.2.3.7.2.4.7.3. 8. Measurement Considerations for Vendors, Testers, and Users 8.1. Measurement Stream Charac. 8.2. Measurement Devices 8.8.1.8.2. 8.3 Measurement Units 8.4. Test Duration 8.5. Clock Sync Options 8.6. Distinguishing Long Delay - Loss 8.7. Accounting for Packet Reordering 8.8. Results Representation and Reporting …8.4.8.5.8.6.8.7.8.8. 12. Appendix on Reducing Delay Variation in Networks 12. 13. Appendix on Calculating the D(min) in PDV 13.

6. Page 6 Recap of activity: IETF-69 and Updates in 04 Strong support for this work and its current conclusions during the IPPM Session  No opposition, either Alan Clark’s comments:  Added ref to G.1050 (and G.1020) in the background  Added new Section 5.5, summarizing metric comparisons in Contribution COM12-D98 (Task: estimate DJB discards)  Percentile based approaches are particularly suitable for SLA agreements. Easy to specify and implement an SLA of the type “no more than x% of packets shall arrive later than y milliseconds” – added to section 6.5  Suggested text for Section 3.5 Separation of scheduling/ smoothing PDV from PDV introduced by the network  Let’s add this if we complete the analysis and find a solution, we have a placeholder now… Stephen Konish Jr.: Support for draft, fix Fig #s sec 6

7. Page 7 Recap of activity: IETF-69 and Updates in 04 (2) Loki Jorgenson’s Comments (much summarizing!): there are implicit assumptions that should be made more explicit, like stream characteristics that are typical and what information is measurable  New paragraph in Background on streams – Neither form is independent of stream characteristics  New section 4.3 – Both forms are 2-point metrics, derived from 1-way Delay (need timestamps)  Addressed in Measurement Considerations Section 8.1 references to "optimizing de-jitter buffer“… motivate the choice of jitter definition - is that related to the (anticipated) mandate of PMOL/APM?  Scope is an Applicability Statement on two forms of IPPM’s metric, using realistic uses for Delay Variation meas.  If this info is applicable elsewhere, so be it. New paragraph in the Scope section 2 gives some cautions.

8. Page 8 Recap of activity: IETF-69 and Updates in 04 (3) Loki’s Comments (much summarizing!): Need more discussion of typical implementation requirements - there is only passing mention of: data processing for statistics with implications on memory req. and reference value D(min), quantiles non available as a running measure. Possibly emphasize G.1020/MAPDV2?  Introduced G.1020 more thoroughly in Background  7.2.1 discusses storage implications, MAPDV2 and RFC3550 Jitter  Added material to Measurement Devices section 8.2 “quick variation” explain as short timescale variation  Explained and replaced in section 5.1 Why no consideration of the practical case where packet arrival times are known but not transmission times for a periodic stream? Does this suggest that there is no meaningful implementation of IPDV without transmission times?  We are comparing the fundamental properties of the DV forms (4.3)  AND, the implications of the definitions on Methods of Measurement – material in Measurement Streams, section 8.1, list item 3

9. Page 9 Recap of activity: IETF-69 and Updates in 04 (4) Loki’s Comments (much summarizing!): Sec. 8.1 Measurement Stream Characteristics, there should be some description of the impact of sampling methodology, where not all the packet D(i) are sampled. This should include some suggestion for methodology references (provided).  Added it, and the references Sec 8.3, Test Duration, some references to the stability of networks would be useful.  Added discussion and references (now 8.4) Sec 8.5, Distinguishing Loss from Delay  Add ref. to draft-morton-reporting-metrics & summary Appendix 12 include equipment & config dysfunction  Another comment suggested to remove this section… Appendix 13 consider including code fragments for the described method

10. Page 10 Recap of activity: IETF-69 and Updates in 04 (5) Carsten Schmoll’s comments: “For me it is clearly the most complete comparison PDV vs. IPDV up to now and very helpful to understand the nitpicks of each definition. (Thanks!) Although PDV is a shifted version of the delay itself, … I believe it has its own justification and use  mentioned in sec 4.2 For IPDV it would be nice to know, how symmetrical it is in practice. Does anyone have results on this?  Packet spacing influences it, see the Ciavattone et al. reference. Esp. for IPDV it is not clear to me the effect of the packet selection function: Does the size of the gaps between selected packets have an influence on the distribution of the resulting IPDV values? (in theory yes (I could give an example), but in practice?)  See the Li and Mills Reference. Conclude with something like a table which recommends clearly the use of PDV or IPDV and suggested stats per desired application. (OK see section 7.3)

11. Page 11 Recap of activity: IETF-69 and Updates in 04 (6) Bob Holley’s comments: IPDV distributions are time dependent – complicates analysis (Agree) Agrees that PDV results are relevant to setting buffer size – and operational perspective From the engineering perspective, the efficiency of queue algorithms is of interest – possible that IPDV can be useful for this (TBD). Symmetrical IPDV dist. means one side is redundant  But this is not assured – see section 5.2 DV>inter-pkt time Clock stability is Demichelis’ strongest argument  See discussion of Clock error in 6.3, IPDV will not have zero mean, so analysis based on that assumption, or that the IPDV distribution is symmetric will be affected.

12. Page 12 Recap of activity: IETF-69 and Updates in 04 (7) Bob Holley’s comments (continued): Did not see how IPDV reacted to the lossless path change.  Brief example added 6.2.2 Path change with Loss is the common case, yet IPDV does not detect it at all! This needs further study.  Do we rely on DV to detect path changes? Does 7.2.2 Frequent Path Changes conflict with this?  Again it depends on what you want to learn from a DV metric. If you want to count path changes with loss, IPDV is not the metric to use. Measurement Units (8.2) discusses implementation  Re-titled Measurement Devices

13. Page 13 Summary of Comparisons Challenging Circumstances for measurement:  IPDV form offers advantages when  Path changes are very Frequent and lossy  Meas. System Clocks exhibit “some” Skew  PDV form is less sensitive to Packet Loss Spatial Composition of DV metric:  All validated methods use PDV,  IPDV sensitivities to sequence and spacing changes is an issue, and tend to break the IID requirement. Estimate of Queuing Time & Variation:  PDV estimates this, especially when sample min = true min Determine De-jitter Buffer Size Required  PDV “pseudo-range” reveals this property by anchoring the distribution at the minimum delay Specification Simplicity (SLA or SLS)  one constraint for PDV single-sided positive distribution

14. Page 14 Summary Table (section 7.3) Comparison AreaPDVIPDV Challenging Circumstances Less sensitive to pkt. loss, simplifies analysis with load balancing & mult paths Preferred when path changes are frequent, or when measurement clocks w/ “some” skew Spatial Composition of DV Metric All validated methods use this form Has sensitivity to seq. and spacing changes & tends to break IID req. Determine de-jitter buffer size required Range or “pseudo- range” reveals this No reliable relation, but some heuristics Estimate of Queuing time and variation Dist. related 1-to-1 on stable path and sample = true min No reliable relation, but some heuristics Spec. Simplicity: Single number SLS One constraint needed for single-sided dist. and rationale as above 2-sided dist., with no summary stat that relates to physical quantity (?)

15. Page 15 Newest Sections and Questions!!! Section 8: Measurement Considerations  This section provides guidance in many areas – Useful?  Do we want to recommend measuring BOTH IPDV and PDV?  Do we want to reconstruct the IPDV and DV later on, with a different interval? Appendix: Guidance for reducing DV in networks  Useful?  One comment at IETF-69 to leave this out

16. Page 16 Summary IPDV and PDV each have their Strengths and Weaknesses Initial (00) Conclusions have seen agreement Suggestions for additional Tasks & Circumstances Need Feedback on Questions & proposed sections and questions raised in the text of the draft Thanks to readers for their comments help finishing this up