1. Carrier Ethernet Technologies and Test Applications Reza Vaez-Ghaemi, Ph.D. November 2008

2. Drivers for Transport Network Evolution
Triple Play services
Business services
Mobile backhauling
Carrier-class performance (SLA guarantees)
Flexible, cost-effective BB Access,
Ethernet and Photonics technologies

3. Carrier Ethernet Requirements
Native Ethernet lacks key capabilities needed for a robust metro core network technology
Ethernet is continuously being enhanced with key features for carrier grade performance such as:
QoS: service differentiation and prioritization
Scalability: granularity and number of services
OAM: Monitoring, loopback
Protection: 50ms path protection
TDM support
OAM
QoS
Protection
Carrier Ethernet
TDM Support
Scalability

4. Basic structure of Ethernet Network
Customer
Provider
Transmission
technique
Customer
Edge
Equipment
Provider
Edge
Equipment
LAN CE PE PB
Packet oriented
Ethernet
Switching
technique
Access
Backbone, Core
WAN, Metro Network
Access

5. Connect networks– Direct, multiple and virtual
LAN CE PE
Eth
Backbone
VPN
VPN realized with native Ethernet
VLAN (802.1p/q); Q-in-Q (802.1ad); MAC-in-MAC (802.1ah)
Switching in the BB by Ethernet transparent switching
VLAN tag serves as filter, not as switching information.
VPN separation only in the access switch
VPN realized with an IP Network
VPLS (Virtual Private LAN Service); MPLS (Multiprotocol Label Switching)
Switching in the BB by MPLS label
VPN separation in the whole BB by LSP (Label Switched Path)

6. VLAN tagging - why VLAN Q in Q
VLAN is a filter function – logical separation of MAC frames
MAC frames are only valid for dedicated egresses
Necessary to block frame distribution to everybody in case of Broadcast function
Used for layer 2 switch based networks
Q in Q
Necessary if customer also uses VLAN tags
Inner tag = Customer tag, Outer tag = Service tag
MAC in MAC Provider Backbone Bridges PBB
Reduces the adress table of provider backbone switches
Only MAC adresses of all provider edge equipment are to be learned

7. Virtual Local Area Network (VLAN) – idea
The idea of VLAN is to segment networks into logical groups rather than physical conditions.
provider device
VLAN #1
VLAN #2
VLAN #3
different offices
management
Logical segmentation
development
controlling
Physical segmentation

8. VLAN configuration and operation – Q-in-Q
802.1ad: Q-in-Q (VLAN-Stacking):
Is necessary, when also the customer is using VLAN in his network.
Customer
C-LAN CE PE
Provider
S-LAN PE CE
Customer
C-LAN
Customer: With VLANs
VID
MAC IP
SVLAN
MAC
CVLAN IP
VID
MAC IP
Switching in the provider net:
SVLAN tag has no switching function.
The SVLAN only hides the CVLAN. The MAC addresses of the customer devices remains visible and are needed for switching.
Still transparent switching and flooding based on the customer Ethernet addresses. This is a big disadvantage in large networks!
Solution: A totally separate (addressing, switching, management) backbone transport network.
PBBN – Provider Backbone Bridge Network (IEEE 802.1ah)

9. MPLS VPLS T-MPLS MPLS-TP

10. G.8112/Y.1371: “Interfaces for the Transport MPLS (T-MPLS) hierarchy”
What is MPLS ?
G.8112/Y.1371: “Interfaces for the Transport MPLS (T-MPLS) hierarchy”
Conceived to be flexibly client of all the relevant transport technologies:
PDH
SDH (POS)
OTN G.709 (GFP)
Ethernet and RPR
Provides service to all types of client traffic: IP
Ethernet
ATM and Frame Relay
PDH and SDH (circuit emulation)

11. Equipment and Terminology
MPLS-VPN
Equipment and Terminology P P C
LSR
LSR C PE PE CE CE
LSR
LSR C P P C
LSR
LSR
C - Customer Router CE - Customer Edge Router
PE – Provider Edge Router P - Provider Router
LSR – Label Switch Router
Customer routers are refered to a C or CE routers and the provider routers facing the customer are called Provider Edge or PE routers and other provider routers are called Provider or P routers. When these routers become MPLS routers they are called Label Switch Routers or LSR. This collection of MPLS LSR’s is called a MPLS Domain.

12. Architecture: The MPLS label...
L3 header
Label
L2 header
Label
Exp S
TTL
32 bits 20 3 1 8
L3 header
Label 1
Label 2
Label 3
Label n
L2 header
Label stack
MPLS label to
ATM VPI/VCI to
Frame Relay DLCI
The MPLS label is inserted between the Layer 2 and Layer 3 header. Multiple labels can be inserted. There are usually at least two labels. The MPLS label can also be used to obtain the VPI/VCI for ATM or the DLCI for Frame Relay. The EXP field is a priority field. This value is either entered manually as part of the configuration for an interface or obtained from the priority field of the Vlan or IP priority fields.
The MPLS label is inserted directly before the OSI layer 3 protocol header. Since the router only evaluates the label, the TTL counter must also be maintained in the label. Several labels can be set as a label stack, and for the next LSR only the top label (last label pushed; in the example, label n) applies: A label is pushed at the start and popped at the end of an LSP.
TTL field with label stacking: If a new label is pushed onto an existing label stack, it gets the current TTL value of the label at the top stack position. If the bottom label in a label stack is popped, the last TTL value of the popped label is copied to all other labels in the stack.
Another major benefit of MPLS is that an edge LSR can set up an LSP even via a VCI in an ATM network or via a DLCI in a Frame Relay network. The LSP is mapped to a VCI or DLCI. In both cases, the switching process is actually shifted to layer 2.
Label: Label value
TTL: Time to live
Exp: Experimental use
S: Stack indicator IP L L2 IP
VPI/VCI IP
DLCI

13. Architecture: Ingress LSR
The domain needs a way in ...
Located at the edge of an MPLS domain
Represents the ingress to an MPLS domain
Assigns packets to an FEC
Inserts the first label
LSR
Edge
LSR
LSR IP IP
LSR
LSR IP
An IP packet enters the Edge LSR without a Label, the PE router or the Provider Edge assign a FEC and inserts the first label. The PE router used to access the MPLS domain is referred to as the Ingress router.
Ingress
LSR

14. Architecture: Egress LSR
… and a way out.
Located at the edge of an MPLS domain
Represents the egress to an MPLS domain
Removes the last label and re-creates the original packet
LSR
Edge
LSR
Edge
LSR IP
LSR IP
LSR IP
The label switched packets traverse the network with Label inserted into them and the outgoing Edge LSR or Egress router removes the Label to transmit an IP packet without a label. The packet enters the MPLS network without a label and exits without a label therefore the CE routers do not need to be MPLS aware.
LSR
Egress

15. The need for VPLS
Switched Ethernet network architectures have proven to be successful in delivering high-performance, low-cost L2 VPN multipoint services.
However, as the size of these switched Ethernet networks has continued to grow, the limitations on the scalability of this architecture has become increasingly apparent. These limitations include:
Limited VLAN address space per switched Ethernet domain
Scalability of spanning tree protocols (IEEE 802.1d) for network redundancy and traffic engineering
Ethernet MAC address learning rate, which is important to minimize broadcast traffic resulting from unknown MAC addresses.
To address the limitations of both MPLS L3 VPNs and Ethernet switching, innovations in network technology for delivering multipoint connectivity services have led to the development of a new technology, which is known as Virtual Private LAN Service or VPLS.

16. What is VPLS ?
Virtual private LAN service (VPLS) is an MPLS application and is used to provide multipoint to multipoint L2 VPN services.
It allows geographically dispersed sites to share an Ethernet broadcast domain by connecting each site to an MPLS-based network.
In contrast to Layer-2 MPLS VPNs, which allow only point-to-point layer 2 MPLS tunnels, VPLS allows a full mesh of sites, allowing any-to-any (multipoint) connectivity.
Assuming our VPLS is MPLS-based, we can use Label distribution protocol (LDP) to create LSPs (mentioned in the previous chapter).
VPLS can also support VLANs on the customer side of a PE, in that case, we must encapsulate a VLAN ID in each packet that is sent

17. VPLS Reference Model
The customer sites are connected through a service provider network, which appears as a Layer 2 switch.
Customer sites are connected to the service provider network at the Provider Edge (PE). All PEs in the network are connected together with each tunnel carrying multiple pseudo-wires. Pseudo-wires are point-to-point connections setup for each offered service between a pair of PEs.

18. MPLS and VPLS – The small difference
VPLS - Virtual Private LAN Service
The customer sees his “own” Layer 2 Ethernet network.
The complete Ethernet packets of the customer are transferred.
PE device emulates to the CE a virtual L2 Ethernet LAN
CE can be a router or an Ethernet switch. PE CE
L2 Eth
L3/IP
+MPLS
MPLS - Multiprotocol Label Switching
The customer sees his “own” Layer 3 IP network.
Only the IP part of the packet is transmitted. Ethernet is terminated at the PE.
PE device emulates to the CE a virtual router.
CE must be a router. PE CE
L3 IP
L3/IP
+MPLS

19. T-MPLS - why
T-MPLS
Makes MPLS a „Carrier Class Network“: Combines the advantages of a reliable packet-based technology (MPLS) with circuit based transport networking.
Based on IP/MPLS technology but simpler implementation: - removes features not relevant for connection oriented applications + addresses critical transport functionality gaps
Connection oriented only
Standardized by ITU-T
Key enhancements: + point-to-point bi-directional LSP + end to end LSP protection (50ms) + advanced OAM support (optimal control of network resources)  lower OPEX

20. Differences MPLS and T-MPLS
MPLS uses uni-directional LSPs T-MPLS uses bi-directional LSPs (it pairs forward and backward LSPs)
MPLS uses PHP (Penultimate Hop Popping): removes the MPLS label one node before the egress - to minimize router processing) T-MPLS doesn‘t use PHP
MPLS uses LSP Merge: all traffic forwarded along the same path to same destination may use the same MPLS label – make OAM and perf.monitoring difficult – it‘s not connection oriented T-MPLS doesn‘t use LSP Merge
MPLS uses ECMP (Equal Cost Multiple Path): it allows traffcic within one LSP to be routed along multiple NW paths – requires additional IP header and MPLS label processing and makes OAM complex T-MPLS doesn‘t use ECMP

21. T-MPLS - Positioning
T-MPLS shifts itself under the packet based networks, in order to organize the “classical” transport networks.
Packet based transport networks: IP Netze
MPLS (= „IP-MPLS“ according IETF)
Data
(L2 & L3 VPN)
Voice
(VoIP)
Internet
(WEB, )
Video
(IPTV, VoD)
Optical transport networks: OTN, WDM, fiber
TDM based transport networks: (PDH), SDH
Packet based transportnetworks: Ethernet, RPR
T-MPLS (= GMPLS without MPLS according ITU-T)

22. Gain of T-MPLS? Network operator Connection oriented
„Carrier Class“ packet switched system for „Carrier Grade“ MPLS
Connection oriented
Management, control and quality is based on proven technologies (e.g. SDH APS)
Convergence of layer 2 (Ethernet) and layer 3 (IP) client signals over one packet based transport technique
With GMPLS one control plane for all network layers (packet, TDM, lambda, fiber)
OPEX and CAPEX lower as with IP MPLS
The question remains: OPEX and CAPEX still lower as with PBB-TE?
Possible responds: Depends on which infrastructure (SDH) is already available and how the further network development is planned.

23. Why MPLS-TP?
Statically configure LSP/PWE via management plane
Deliver OAM for LSP/PWE
Consistent OAM for multi-layer networks which enables interworking of services such as LSP, PWE, and L2
Offer MPLS LSP and PWE as a transport service
Manage LSP/PWE at nested levels (path, segment)
Additional protection switching
Congruent OAM and Traffic including LAG/ECMP

24. What is MPLS-TP?
Definition of an MPLS Transport Profile (TP) within IETF MPLS standards
Based on PWE3 and LSP forwarding architecture
IETF MPLS architecture concepts
Major concept is LSP.
PW is a client.

25. LSP and OAM A segment is between MEPs
• OAM is end to end or per segment
• In SDH/OTN and Ethernet segment
OAM is implemented using Tandem Connection Monitoring (TCM)
• The OAM in each segment is independent of any other segment
• Recovery actions (Protection or restoration) are always between MEPs i.e. per segment or end to end

26. Associated Channel Level (ACH)
Generalised mechanism for carrying management / OAM information
OAM capabilities : Connectivity Checks (CC) and “Connectivity Verification” (CV)
Management information: Embedded Control Channel (ECC)
Data Communications Network (DCN)
Signalling Communication Network (SCN)
APS information

27. How does it work?
Processed by the pseudo-wire function on the end-points
End point or Pseudo-wire stitch point
Verifies the operational status of the pseudo-wire
Working with the native attachment circuit technology
An inter-working function with the native attachment circuit OAM.
Transport and act upon native attachment circuit OAM technology

28. PBB
PBT/PBB-TE

29. Why PBB? Eliminates address space scalability issue
Solves MAC table size issue
Works in conjunction with 802.1ad (Q in Q)

30. What is PBB?
A PBBN is its own virtual bridged local area network, which is completely under the administrative control of the backbone provider.
Switching: Transparent switching and flooding based on the PE MAC addresses.
By introduction of a further MAC layer (MiM) a clear separation between customers and provider results to addresses, switching and management.
For data separation, PBB adds an I-tag and a B-tag.
Customer data (MAC addresses, VLAN tags) are transparently transported.
Multiple spanning tree enables to distribute the backbone load to several even parallel paths. Normally a MST per VLAN.
MAC Addresses of PE switches
Address cache
Port 1: W
Port 2: X
Port 3: Y,Z
MAC(A,B,C)
MAC(S,T,U)
(W)
(Y)
MAC(H,I) 1 3
MAC(J,K,L) 2 PB
(X)
(Z) PE
BB-Access
Backbone
Access B U B W Y U I B U

31. PBB – 802.1ah detailed frame format
FCS
B-SA
B-DA
TPID
B-VID
B-DEI
B-PCP
I-SID
I-DEI
I-PCP
RES1
RES2
Payload 48 16 3 1 12 2
24(120) 32
B-TAG
B-TCI
I-TAG
I-TCI
B-DA, B-SA: Backbone Destination und Source Address
TPID: Tag Protocol Identifier (Indication, which TCI follows. C-, S-, B, or I-TCI)
B-PCP: Backbone VLAN Priority Code Point (Shows the CoS for prioritized transport.)
B-DEI: Backbone VLAN Drop Eligible Indicator (Shows whether the frame can be preferred
B-VID: Backbone VLAN ID dropped in overload situations.)
B-TCI: Backbone TAC Control Information
I-PCP: Service Instance Priority Code Point
I-DEI: Service Instance Drop Eligible Indicator
I-SID: Service Instance Identifier
RES1/RES2: Reserved for future applications

32. PBB/PBB-TE Test
Terminating PBB trunks, testers T1 and T2 perform an end-to-end layer 2 test configured with B-Tag fields in Mac in Mac format.
Testers only filter on B-MAC/B-VID/(I-SID) and handle everything else in the frame as part of the payload. T1 T2
BCB-BEB Interface
BCB-BEB Interface
Customer 2 Site B
BCB
BCB
Customer 1 Site A PB CB CB PB
BEB
BEB CB CB
BCB
BCB PB PB
Customer 1 Site B
Customer 2 Site A

33. Ethernet OAM

34. Services Connectivity Transport/Link Ethernet OAM Layers
ITU Y.1731 and MEF
Services
IEEE 802.1ag, ITU and MEF
Connectivity
EFM (IEEE802.3ah)
EoSDH (ITU)
EoTDM
(ITU)
Transport/Link

35. Ethernet OAM Standards Activities
UNI
UNI
100FX
Carrier NW
100FX
Media Convrtr
Media Convrtr
Carrier Edge
Carrier Edge
Media Convrtr
Media Convrtr
802.1aj demarcation device
802.1aj demarcation device
MEF &
ITU Y.1731
Service Layer OAM (UNI to UNI)
IEEE 802.1ag,
MEF & ITU Y.1731
Connectivity Layer OAM
Access Link OAM
Access Link OAM
IEEE 802.3ah

36. Ethernet OAM Standards
Services and Performance
(ITU Y.1731/MEF)
Basic Connectivity
(IEEE 802.1ag,ITU)
Transport/Link
(802.3ah EFM)
Discovery
Continuity check (keep alive)
Continuity check
Remote failure indication: Dying gasp, link fault & critical event
Loopback (non-intrusive and intrusive)
Loopback
Remote, local loopback
AIS/RDI/Test
AIS
Fault isolation
Link Trace
Performance monitoring with threshold alarms
Performance management
Status monitoring
MRV 1m
I think it’s worth pointing out some of the specific OAM functions that 802.3ah does and does not support. The table on the screen breaks these down into several key categories with specific examples listed within each. The thing to note, here, is that even where 802.3ah may not support a particular OAM function, an Ethernet Access equipment vendor may implement its own extension to the standard and include support for that function without compromising interoperability. It is this flexibility of the 802.3ah standard that makes it so powerful.
Now we won’t take the time to discuss each item listed in the chart. Must of these I’m sure you are already familiar with. However, if there is one or two that you would like to go over in more detail, just send Andy a message via the chat window and we’ll try to cover it during the Q&A session at the end of this presentation.

37. Management Layer Comparison
Ethernet
Link/Domains
Link
Loopback
RDI
Event Monitoring
Signal/line quality (frames/symbols)
Domains
CFM Loopback (“ping”)
CFM Linktrace (“traceroute”)
CFM Connectivity Check
SONET
Line/Section/Path
Line
Loopback
RDI
Signal/line quality (BER)
Section
Path

38. Ethernet First Mile OAM
Why not just use IP-based management (SNMP, Telnet, etc.)?
IP infrastructure must be operational and properly configured
Some networks don’t use IP
Security vulnerability
IEEE 802.3ah OAM
Provides low level Ethernet OAM functionality at FM
Isolated to First Mile segment
Complements SNMP and other IP-based NMS
NMS can be secure and away from the user
Media independent: Fiber (Active P2P or PON), Copper
Uses standard Ethernet frames – slow – 10 frames/sec
Backward compatible with non-802.3ah Ethernet
Requires minimal configuration (almost “plug and play”)

39. Ethernet First Mile OAM
(Access Link)
First Mile
(Access Link)
Carrier
Network
802.3ah
802.3ah
NMS
Defines:
EDD – Ethernet Demarcation Device (NID)
Functions:
Monitor Link performance
Fault detection and notification (signaling)
Loopback testing

40. Ethernet 802.3ah Protocol
First Mile
(Access Link)
First Mile
(Access Link)
Carrier
Network
802.3ah
802.3ah
NMS
Discovery – Exchange of capabilities and i.d. info
Periodic state update / monitoring
Link event notifications (performance threshold alarms)
Remote failure notification –
Link faults (Network side port) including Unidirectional
Dying Gasp – catastrophic / non-recoverable failures
Critical events – vendor specified events
Remote Loopback
Read MIB info / stats
Vendor extensions

41. Management of Network Edge
UNI
MAC
Device
Service Provider PE
802.3ah OAM
MEF ELMI
802.1aj TPMR
Multiple standards provide building blocks for managing the access
802.3ah OAM addresses how to manage physical from the PE to the MAC layer of the remote device
MEF ELMI addresses how to manage the UNI of the remote device
802.1aj Two-Port MAC Relay addresses how to manage a potential customer demarcation device
Each protocol adds some unique functionality, and (unfortunately) adds some overlap functionality with the other standards

42. IEEE 802.1aj TPMR IEEE 802.1aj Two-Port MAC Relay
Defines protocols for interactions carrier bridge and remote 2-port relay device that might be used for demarcation
Uses SNMP natively over Ethernet (no IP!)***
Configuration of remote parameters***
VLAN behavior
CoS and QoS characteristics of interfaces
Forwarding behavior (data and L2 protocols)
Status information between carrier and customer***
Utilizing IEEE 802.1ag
Covers configuration and status of all aspects of remote device IF the device is a simple two-port relay forwarder
Does not apply to bridges, routers, etc. which may also implement UNI or forwarding functionality
Mostly intended for CARRIER owned demarcation device
***Tentatively

43. Ethernet Service OAM
Being standardized by Metro Ethernet Forum and ITU (Y.1731)
Builds upon IEEE 802.1ag functions by utilizing timestamps and other fields to monitor SLA metrics for Ethernet services
Measures latency, jitter, loss, etc. end-to-end across any domain
Used to validate SLA performance of Ethernet service across any kind of underlying network

44. Ethernet OAM Implementation
802.3ah
(Access Link)
Operator A
Operator B
802.1ag/ Y.1731
(UNI-N to UNI-N ME)
Focus on end-to-end OAM (IEEE802.1ag, ITU Y.1731)
incorporating last segment OAM (IEEE802.3 ah)
End-to-end service OAM:
Continuity Check
Loopback
Performance Monitoring
Frame Delay, Frame Delay Variation, Frame Loss, Availability

45. Customer End-to-End Metrics Provider End-to-End Metrics
IEEE 802.1ag CFM
CFM = Connectivity Fault Management (in progress)
Partitions network into hierarchical administrative domains
Basic connectivity checking and troubleshooting across any domain, and across multiple domains at the same time
Partitions big problem into pieces & controls visibility
Service Provider
Operator
Operator
Customer End-to-End Metrics
Provider End-to-End Metrics
Operator Metrics
Operator Metrics

46. IEEE 802.1ag CFM
Providing “ping” and “traceroute” equivalents for bridged (rather than routed) networks
Loopback (= Ping): Given MAC address and VLAN tag, verify connectivity
LinkTrace (= Traceroute): Given MAC address and VLAN tag, find the path between the local source and that destination
Also provides continual connectivity checking (a.k.a. heartbeat) to proactively monitor end-to-end availability and packet loss

47. Continuity Check (CC) OAM
Operator A
Operator B
CC Sink
CC OAM Frames
CC Source
CC OAM Defined per EVC, one-way connectivity monitoring
Loss of Continuity (LOC) is declared (only at the sink side) upon 3.5 seconds without receiving CC OAM frame
Loss of Continuity (LOC) is cleared upon sink receiving 2 CC OAM frames within a window of several seconds
Upon CC failure
Send trap
Update active alarm log
Update statistics
Optional uplink switch-over

48. Loopback (LB) OAM
Operator A
Operator B
LB OAM Request
MEP
MEP
LB OAM Reply
Per Link/Port: Remote loopback minimizes a false repair calls
Non-intrusive loopback: Defined per EVC/service monitoring
one VLAN is looped back while others continue to provide their services
LB Failure is declared upon 2 sec without receiving OAM frame
Upon LB failure
Send trap; Update active alarm log; Update statistics
Optional uplink switch-over

49. Performance Monitoring (PM) OAM
Defined per service (EVC+CoS), allows for in-service, end-to-end SLA monitoring
Measured parameters
Frame Delay, Frame Delay Variation, Frame Loss, Availability
Performance Measurements traps
Traps sent upon: {# Frames} crossing {Objective} within {Sampling Time}
The above applies for both ‘Rising’ and ‘Falling’ thresholds definition
Statistics are collected per 15 minutes intervals; Cyclic 96 intervals kept (24 hours)

50. Test Application Field
Analyze/Generate OAM errors/alarms
Performance Measurements
Loopback tests
UNI
Media
Convrtr
Carrier
Edge
IEEE 802.3ah
MEF &
ITU Y.1731
Access Link OAM
IEEE 802.1ag,
MEF & ITU Y.1731
Connectivity Layer OAM
Service Layer OAM (UNI to UNI)
100FX .
Carrier NW
802.1aj demarcation device