1. MB – NG SuperJANET4 Development Network
Production
Network
Leeds
RAL /
UKERNA
SJ4 Dev
C-PoP
Reading
12416
Warrington
Manc
MCC
ULCC
UCL
London
Gigabit Ethernet
2.5 Gbit POS
2.5 Gbit POS core
WorldCom
Dark Fibre (SSE)
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

2. Some Edge (Bbone) Router requirements
Connect to the test systems in the IP domain.
Accept marked packets and be able to re-mark them
Mark the packets on input according to at least:
Source – dest IP address IP port
Layer3 and Layer4 headers TCP/UDP
Support Diffserve code points
Perform policing and admission control
Provide a range of Queue type options –
Several queues with different priorities
Real-time queue
Provide flexible Queue Scheduling
Weighted round robin
Weighted fair queuing
Provide Congestion control such as
RED
ECN
Be able to assign MPLS labels to the packets in a flexible manner.
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

3. Equipment for the External / Edge Domains Functionality
Load PC
Test PC
Difserve router OR
MPLS Edge Router
4 * 1 Gigabit Ethernet blade
QoS Marking
Gigabit Ethernet
2.5 Gbit POS
2.5 Gbit POS core
SJDN Core
MPLS Backbone Router 12416
3 * 1 Gigabit Ethernet blade
10 Gbit POS
MPLS
MPLS Backbone Router OSR
IP or
Supervisor
Classify packets
QoS Difserv
QoS – WRED
MPLS
Generate MPLS frames
Multi-loop
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

4. Equipment for the External / Edge Domains Connectivity
Gigabit Ethernet
2.5 Gbit POS
2.5 GbitPOS core
Spy
3508
Monitor PC
Load PC
Test PC
Ethernet Switch 3512
Local site Ethernet
Test PC
4 * 1 Gigabit Ethernet blade
MPLS
4 * 1 Gigabit Ethernet blade
QoS Marking
Test PC
Test PC
Load PC
MPLS to core
2.5 Gbit POS
MPLS
Load PC
Supervisor
Supervisor
Load PC
Difserve router OR
MPLS Edge Router
MPLS Backbone Router OSR
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

5. IP Access to MB-NG Systems A suggestion only – we all must decide !
End/Test systems routing:
The default route from the PC is set to the Gigabit link i.e. into the development network.
Static routes for researchers only are set for the 100 Mbit interface, e.g. to Manchester HEP, MCC, RAL, UCL, ULCC, UKERNA.
The link between the Cisco 3512 and the Campus is set to 10 Mbit – this would minimise any unintentional traffic.
IP access to core routers for researchers only
Similar approach? via the test PCs at the CPoP
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

6. R. Hughes-Jones Manchester
Some Questions ???
Can one Gigabit Ethernet blade perform the IP and MPLS functions we need or do we require two cards – one for IP input functions, the other for MPLS output?
 What is the performance of the 4-port Gigabit Ethernet interfaces?
Confirm that an interface can support both IP and MPLS frames at same time.
Can the GSRs use MPLS to loop the frames round n times?
Can all our “requirements be” met?
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

7. Focus on the routers - Hardware
Cisco OSR based on 6500 Are the details correct ?
More memory
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

8. Focus on the routers - Summary
Focus on the routers - Summary
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

9. Hardware & Software Tools
High Bandwidth High Throughput Protocol Investigations (mainly)
4 * 1 Gigabit Ethernet blade
Test PC
2.5 Gbit POS
Monitor PC
NIC
EthernetSwitch
3508
Monitor port
4 * 1 Gigabit Ethernet blade
Test PC
2.5 Gbit POS
Monitor PC
NIC
Optical splitter or opto-electrical tap
opto-electrical tap constructed for logic analyser
Software use of :
tcpdump tcptrace
Web100
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

10. Latency Tests with 64-bit PCI
Alteon Gigabit NIC + RAW
SuperMicro 370DLE
ServerWorks III LE Chipset
64 bit 33 MHz
Request – Response
Transfer rates:
observe expect us/B
32+32 bit
-> 108Mbyte/s
32+64 bit
64+64 bit
64 byte Request
1400 byte Response
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

11. Throughput Tests with 64-bit PCI
RAW Stream
Not using IP stack
Vary time between packets
1.6 factor improvement over bit transfers
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

12. Throughput Tests UDP SysKonnect
UDP Stream
Vary time between packets us
No interrupt coalescing
15 μs between packets
1400 byte sent
1400 byte received
Pci-pci latency 28.3 μs
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

13. Latency Tests UDP SysKonnect
UDP request - response
Latency and transfer rates very similar to raw frames
observe us/B offset us
64+64 bit
64+64 bit
(raw)
1400 byte sent
10.66 μs pci time
1400 byte received
10.96 μs pci time
Pci-pci latency 28.3 μs
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

14. Throughput Tests UDP Alteon
UDP Stream
Vary time between packets us
interrupt coalescing
15 μs between packets
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester

15. Latency Tests UDP Alteon
UDP request – response
Due to interrupt coalescing
MB-NG Meeting Dec 2001
R. Hughes-Jones Manchester