1. A Deterministic End to End Performance Verification Architecture
Jerry Sobieski
NORDUnet
October 20, 2012

2. The Problem
Emerging Connection Services that offer “guaranteed” performance require a means of:
A) determining if a Connection is performing as requested
B) determine where (and why) a Connection is failing
Guaranteed service require substantially

3. Deterministic Performance Verification
Can we deterministically measure the performance of real traffic across a domain?
Can we do so without perturbing the flow?
Can we do so in such a fashion that we can determine where along the path performance problems are occuring?
Network domain
STP-A
STP-Z
How do we verify the throughput from STP-A to STP-Z?

4. How do we measure traffic across a domain?
Existing models pose specialized performance measurement servers
Active measurements are performed that replace or perturb existing flows – the flows we want to understand!
Traffic characterization can only be measured between these specific Measurement Points (MPs).
the MP to MP path often includes path segments and/or network elements that are not directly part of the path of interest
Some tests often transit components with indeterminant performance characteristics
E.g. a “Ping” incurs indeterminant latency based upon processor loads.

5. What do we need to do better?
Passive measurement
Measure the actual user data flows instead of artificial flows
Consistent measurement points
Architecturally predictable measurement locations.. Everywhere.
Architecturally deterministic measurement points themselves… simple, cheap, and effective.
Highly accurate timing
What resolution is required to characterize 100 Gbps packet flows?
Appropriate software tools
Capture/analysis
Automated
User and/or Operations oriented
Secure

6. A “Performance Flow Correlator”
PV Flow
correlator 4 5
Flow sample
(Realtime/background,
step/full sampled) 3
Ingres Flow “tap”
(splitter/hdw replication) 2
Circular Flow buffer
(time stamped capture servers) 1
Ingress flow
Egress flow
STP-1
STP-2
user flow
Border
switch
Border
switch
External
domain
External
domain
Intra-Domain
Flow Correlation

7. How the Flow Correlator Works
The “flow tap” design can be implemented at every domain boundary, at every interface.
Such mirroring capabilities are often already incorporated in switching interfaces – just never used for systematic operational monitoring and fault localization
Optical Passive splitters can be easily inserted inline at the interface where the device does not have such capabilities
Electrical signals can also be tapped using mirror ports.
The tap, when enabled, leads directly to a local flow buffer.
The flow buffer can be sized and configured to capture an entire flow, or it can be sized to sample flows according to some rule or policy.
The correlation can be done in real time if engineered to do so…
or the flow can be captured and stored for later background analysis. … a few seconds later, or a few days later.
Correlation can be performed periodically, using short samples or real flows

8. An inter-Domain “Flow Correlator”
PV Flow
correlator
Flow samples are FTP’d (background processing)
or streamed (real time processing) to correlator
The inter-domain flow transfer Should use dedicated
circuits to avoid affecting other traffic
Inter-Domain Transport Infrastructure
Flow
buffer
Flow buffer
Egress flow
Ingress flow
STP-A
user flow
STP-Z
Border
switch
Border
switch
Inter-Domain
Flow Correlation
Source
Domain
Destination
Domain

9. End to End PV Architecture
Aruba
Bonaire
Curacao
Dominica
MP-B1
MP-C2
MP-D1
MP-D2
MP-A1
MP-A2
MP-B2
MP-C1 B
Stp Z
Stp A D C
This data plane architecture can deterministically localize faults to a particular domain.
Automated agents can perform the analysis
NSI Query() primitive can be used to provide both the multi-domain path and the the boundary STPs
A simple directory can map STPs to Flowbuffers
Or an active agent (NSA?) can be requested to validate path performance.

10. A look at the border switch
Each STP must be able to mirror its specific data stream to the flow buffer
Thus, physical channels must be tapped (fiber, UTP, etc.)
And virtual channels must be tapped (vlans, LSPs, TDM channels, etc.)
The tap should be placed as close to the ingress/egress STP as possible.
Outside of elements such as buffers or framers
Doing so provides for better fault localization.
The mirrored flow must preserve performance fidelity between the tap and the flow buffer capture interface
(i.e. no mirror aggregation or buffering prior to capture.)
The flow buffer should be sized to process the number and size of simultaneous flows it will be expected to process

11. Timing
Simple packet recording at source and destination can provide accurate packet loss insights even without timing
But not performance fidelity
Inter-packet arrival times can be measured accurately locally at each end
Can reveal jitter characteristics after correlation
Presta packet capture cards can do 10 nS time stamping at 10 Gbps.
These cards need an external [accurate] clock
Other similar cards can also do packet filtering and other intelligent offloading.
Could be useful for sourcing streams with very accurate pacing, or payload signatures.

12. Highly accurate distributed timing
Emerging techniques in metrology allow extremely accurate timing to be distributed though the photonic network.
Accurate to 10^-17 seconds (!)
Advanced NICs can be field programmed to time stamp packets to 10 nanoSecond resolution
Driven by external [distributed] clock
A 64 byte packet arrives every 512 nS on a 1 Gbps link
A 64 B packet arrives no more than every 51 nS on a 10 Gbps link.
E2E packet Latency can be computed with distributed global clock to 10 nS intervals…for each packet in a large flow (!)

13. Interdependence
NSI SDN