1. Establishing End-to-End Guaranteed Bandwidth Network Paths Across Multiple Administrative Domains
The DOE-funded TeraPaths project at Brookhaven National Laboratory investigates the combination of DiffServ-based LAN QoS with WAN MPLS tunnels and dynamic circuits in establishing true end-to-end (host-to-host) virtual paths with bandwidth guarantees. These virtual paths can be associated with specific data flows and are managed through advance reservations, providing the capability of network resource scheduling. Scheduling network resources is critical in a data-intensive computing environment such as that of the ATLAS experiment.
WAN
Minimum Best Effort traffic
Dynamic bandwidth allocation
Shared dynamic class(es) Dynamic microflow policing
Mark packets within a class using DSCP bits, police at ingress, trust DSCP bits downstream
Dedicated static classes Aggregate flow policing
Shared static classes Aggregate and microflow policing
TeraPaths
Site A
Site B
Site C
Site D
WAN 1
WAN 2
WAN 3
service invocation
data flow
peering
WAN chain
WAN
WAN web services
TeraPaths 1 2 3
The TeraPaths software can partition a site’s available network bandwidth into multiple classes of service with statically or dynamically assigned bandwidth.
Virtual paths are stitched together by directly configuring end site LANs and interfacing with WAN services to provision a suitable, “DSCP-friendly” path between these end sites.
WAN
border router
MPLS tunnel
ingress/egress
router
WAN
switch
border router
The project has successfully developed techniques to configure LAN devices while satisfying site security requirements. TeraPaths can utilize both L3 and L2 connections. L2 connections (dynamic circuits) require proper VLAN configuration at the end sites and Policy Based Routing (PBR) within a site’s LAN for VLAN selection. A secondary effect of dynamic circuit utilization is per-data flow route selection in addition to guaranteed bandwidth.
The TeraPaths software is fully distributed. A site instance comprises a set of web service modules. TeraPaths services are accessible via both a web interface and an API, which can be used by 3rd party clients.
current
US ATLAS T2 sites
Internal Services
Public Services
Web Interface
Admin Module
NDC
• • •
Database
protected network
API
remote
local
WAN Services
proxy
CLI
s/w client
LAN/MPLS
TeraPaths
resource
manager
MPLS requests
traffic identification:
addresses, port #, DSCP bits
grid AAA
Bandwidth Requests & Releases
OSCARS
ingress / egress
LAN QoS
M10
data transfer
management
monitoring
GridFtp & dCache/SRM SE
network
usage
policy
ESnet
remote
Remote LAN QoS requests
The TeraPaths testbed currently includes QoS-enabled subnets at BNL and at the University of Michigan and is being expanded to all US ATLAS Tier 2 sites. These sites are interconnected through ESnet and Internet 2. The testbed enables valuable networking research for the benefit of the Physics community and beyond.
CHEP 2007