1. 1 Benchmarking Methodology WG (bmwg) 73rd IETF – Minneapolis, MN USA Monday, November 17, 2008, 1300-1500 Afternoon Session I (Rochester) Chairs: –Al Morton (acmorton@att.com) If you are not subscribed to the BMWG mailing list and would like to be, put a BIG asterisk (*) by your legibly-printed e-mail address when you sign the blue attendance sheet, or go to https://www1.ietf.org/mailman/listinfo/bmwg

2. 2 BMWG Agenda (Any Bashing needed?) Note-Taker(s), Jabber, IPR (Yellow Sheet), Blue Sheets WG Status (Chair, 10 min) IGP Dataplane Convergence Status (Poretsky, 20 min) Protection Benchmarking (Poretsky, 20 min) SIP Device Benchmarking (Poretsky, 20 min) Milestones Status (Chair, 5 min) Proposed Work (Chair, 40 min, or time remaining) –IP Flow Info Accounting and Export Benchmarking (Claise for Novak) (not appearing this time): –LDP Convergence (Asati) –Wireless LAN Switches Alexander) –Multicast VPN Scalability (Dry) AOB (???, 5 min)

3. 3 Charter Text Extension In addition to its current work plan, the BMWG is explicitly tasked to develop benchmarks and methodologies for the following technologies: * MPLS Forwarding: Develop specific methods to characterize the latency and forwarding performance of MPLS devices, extending the fundamental recommendations of RFC 1242 and RFC 2544 to this networking technology. * SIP Networking Devices: Develop new terminology and methods to characterize the key performance aspects of network devices using SIP, including the signaling plane scale and service rates while considering load conditions on both the signaling and media planes. This work will be harmonized with related SIP performance metric definitions prepared by the PMOL working group.

4. 4 BMWG Activity AD/IESG Review Rev I-Ds Needed WG Last Call draft-ietf-bmwg-ipsec-meth-03.txt Rev I-Ds Needed draft-ietf-bmwg-ipsec-term-10.txt Then another WGLC Rev I-Ds Needed Then another WGLC I-Ds draft-ietf-bmwg-protection-term-05.txt Discussion Today, then draft-ietf-bmwg-protection-meth-04.txt WGLC ??

5. 5 OLD ACTION ITEMS IPsec I-D announcements should be posted to the IPsec mailing list. Post the LDP-convergence drafts to the MPLS WG so that we wouldn't repeat Scott's experience with the IGP drafts (*Authors* are asked to do this when they are ready, since this isn't a bmwg work item yet). Also, consider the LDP-IGP sync issue (has not been fully addressed in the protocol dev wgs). Regarding the non-SIP specific metrics in the SIP draft, a possible resolution may be to be to revise the names of the benchmarks to make them SIP-specific (especially if they are anchored on SIP protocol events).

6. 6 BMWG Activity Candidates for Chartered I-Ds –draft-poretsky-sip-bench-term-04.txt <= Revised, consider 4WG –draft-poretsky-bmwg-sip-bench-meth-02.txt <= ?? Expired ?? Work Proposals. –draft-karthik-bmwg-ldp-convergence-meth-02.txt <= Need IGP prog. –draft-eriksson-ldp-convergence-05.txt <= Need IGP prog. –draft-alexander-bmwg-wlan-switch-term-01.txt <= Just expired –draft-alexander-bmwg-wlan-switch-meth-01.txt <= Need Readers –draft-vapiwala-bmwg-TE-convergence-motivation-01.txt <= Expired –draft-sdry-bmwg-mvpnscale-03.txt <= Just expired –draft-salahuddin-bmwg-bfd-motivation-00.txt <= Expired –Draft-novak-bmwg-ipflow-meth-00.txt <= NEW!!! Expected I-Ds –

7. 7 BMWG Activity Expired BMWG I-Ds –, – Pending term – RFC Editor Queue – New RFC: – Charter Update – none Supplementary BMWG Page –See http://home.comcast.net/~acmacm/BMWG/

8. 8 Standard “Paragraph” (intro/security) Benchmarking activities as described in this memo are limited to technology characterization using controlled stimuli in a laboratory environment, with dedicated address space and the constraints specified in the sections above. The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network, or misroute traffic to the test management network. Further, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the DUT/SUT. Special capabilities SHOULD NOT exist in the DUT/SUT specifically for benchmarking purposes. Any implications for network security arising from the DUT/SUT SHOULD be identical in the lab and in production networks.

9. 9 Additional Standard Sections or Paragraphs for BMWG memos? Scope (plus purpose and application) –ALL RFCs really need this, and yet… Test Device Capabilities –Starting to see some value for this in the IGP Dataplane work –Specific capabilities needed –Areas where we challenge the Equip Vendors Others?

10. 10 Current Milestones Sep 2008IPsec Device Benchmarking Terminology to IESG Review Sep 2008IPsec Device Benchmarking Methodology to IESG Review Dec 2008Net Traffic Control Benchmarking Methodology to AD Review. Dec 2008Router Accelerated Test Terminology to IESG Review Dec 2008Router Accelerated Test Methodology to IESG Review Feb 2009Terminology For Protection Benchmarking to AD Review Feb 2009Methodology For Protection Benchmarking to AD Review Apr 2009Methodology for MPLS Forwarding to AD Review Jun 2009Terminology for SIP Device Benchmarking to IESG Review Jun 2009Methodology for SIP Device Benchmarking to IESG Review Dec 2009Router Accelerated Test Method for EBGP to IESG Review Dec 2009Router Accelerated Test Method for Op Sec to IESG Review Jul 2010Basic BGP Convergence Benchmarking Meth. to AD Review.

11. 11 Work Proposal Summary Matrix Work Area > Criteria \/ Mcast VPN Scalability LDP Converg WLAN switch IPFIXBFD RSVP- TE Firewall extension Proposal YYYYYYY In Scope of Charter? (acm) YY No Overlap w/802.11 T YYYY Draft(s) 1+1+1 1+ Draft gives motivat. ? Sig. Support at meetings Yes, but also some objections Sig. Support on List Many 12/2007 comments Many 2/2008 comments Some comments Questions to clarify Dependencies L3vpn specIGP prog