1. Real-world Requirements Pitfalls of
Segment Routing Traffic Engineering
Timmy Schüller
Osnabrück University
Last year at RIPE, Florian Hibler (from Arista) gave a talk about SR and why it is cool. I agree, but I found that it also breaks things, if you use it for TE and are not careful.
RIPE 78
Reykjavik, May 21, 2019

2. Traffic Engineering using Segment Routing
.. is a Source Routing Architecture
.. uses Node, Adjacency and Service Segment IDs to define “checkpoints”
.. encodes a path as a stack of SIDs
.. consults IGP to reach an SID
 SR tunnel ≔ concatenation of shortest paths
.. can be implemented with MPLS or IPv6 & an IGP extension O
First, let me remind you on what SR is. E S U U C U U D B O S
Data A F S S O O
Copyright © 2019 Timmy Schüller, Osnabrück University

3. 2SR .. 𝑘 𝑘 𝑘 𝑖 𝑗 𝑖 𝑗 𝑖 𝑗 𝑖 𝑗 ℓ ℓ ✔ ✔ ✔ ✔ ✘
2-Segment Routing
2SR ..
.. minimizes max. link utilization
.. Uses at most 2 concatenated shortest paths per SR tunnel ✔ 𝑘 𝑘 𝑘
50% 𝑖 𝑗 𝑖 𝑗 𝑖 𝑗 𝑖 𝑗
50% ℓ ℓ ✔ ✔ ✔ ✘
first practical issue is the size of the label stack. But in practice, 2 segments suffice [google maps analogy]
Copyright © 2019 Timmy Schüller, Osnabrück University

4. Real-World Requirements I
Minimize max. utilization
Image by Ian Flickr
This is the main goal of TE (and most common objective in literature)
Copyright © 2019 Timmy Schüller, Osnabrück University

5. Output Input Max. util. = 64% Use SR Paths: RKV  OS  FRA
2SR by Bhatia et al.
Image from (Accessed June 2015)
Max. util. = 64%
Output
Input
Use SR Paths:
RKV  OS  FRA
FRA  OS  RKV
etc.
Source
Destination
Traffic
RKV
FRA 12 OS 7 42
etc.
To min max. util with 2SR, we can just take the linear program by Bhatia et al. At INFOCOM 2015
Input: Topology and Traffic Matrix Output: Objective value and how to get there (SR configuration)
2SR formulation from Bhatia et al.: Optimized Network Traffic Engineering using Segment Routing. IEEE INFOCOM, 2015
Copyright © 2019 Timmy Schüller, Osnabrück University

6. 10 Snapshots of a Tier 1 ISP Backbone from 2018
Verifying 2SR
10 Snapshots of a Tier 1 ISP Backbone from 2018
Some trace-based simulation results that verify that 2SR works well
lower bound
state of the art
Copyright © 2019 Timmy Schüller, Osnabrück University

7. Real-World Requirements II & III
Minimize max. utilization
Keep # of SR tunnels low
Limit split factors
Image by Ian Flickr 𝑘 1% 𝑖 𝑗
99% ℓ
Now we take our result, go to operations and tell them to deploy this specific SR configuration. Unfortunately they are not happy, because it contains thousands of SR paths.. Also, apparently, a router is not capable of splitting traffic in arbitrary fractions, which the presented model does
The Tunnel Limit Extension (TLE)
Copyright © 2019 Timmy Schüller, Osnabrück University

8. Real-World Requirements IV
Minimize max. utilization
Keep # of SR tunnels low
Limit split factors
Avoid transit on edge nodes
Image by Ian Flickr ✘ ✔ 𝑘 1% 𝑖 𝑗
99% ℓ
We showed the resulting SR configuration to the customer, but they identified illegal paths
that used virtual edge nodes as intermediate segment
Copyright © 2019 Timmy Schüller, Osnabrück University

9. With this modular design, any node may be blacklisted!
Defining a Blacklist
Step 1: Define nodes to be blacklisted (e.g. ‘*Edge’)
Step 2: Automatically reorder list of nodes:
Step 3: Add constraint to optimization: ✘ ✔
Osnabrück A
Osnabrück B
Osnabrück Edge
Reykjavik A
Reykjavik B
Reykjavik Edge
Frankfurt A
Frankfurt B
Frankfurt Edge
Osnabrück A
Osnabrück B
Reykjavik A
Reykjavik B
Frankfurt A
Frankfurt B
Osnabrück Edge
Reykjavik Edge
Frankfurt Edge
Nodes that may be used as checkpoints
Nodes that must only be source and/or destination
To avoid undesired transit traffic, we defined a blacklist.
The blacklist is freely configurable, any node may be blacklisted.
Disclaimer: Examples are purely fictional
With this modular design, any node may be blacklisted!
Copyright © 2019 Timmy Schüller, Osnabrück University

10. Real-World Requirements V
Minimize max. utilization
Keep # of SR tunnels low
Limit split factors
Avoid transit on edge nodes
Avoid bypassing latency requirements
Image by Ian Flickr ✘ ✔ 𝑘 1% 𝑖 𝑗
99% ℓ
While experimenting, weird flows appeared that bypass latency requirements, since intermediate segments can be placed freely!
Copyright © 2019 Timmy Schüller, Osnabrück University

11. Intracontinental Delay Constraint:
Delay Constraints
Intracontinental Delay Constraint:
Intranational Delay Constraint:
We cannot use our blacklist for this, because nodes need to be blacklisted given the context of the traffic flow
This is not a perfect solution, but approximates the issue well enough
(actually readable math: if continent of ingress node is the same of the continent of the egress node AND the continent of the intermediate router is NOT the same as the continent of the ingress, set traffic variable to zero)
Intra-Site Delay Constraint:
Copyright © 2019 Timmy Schüller, Osnabrück University

12. The Tunnel Limit Extension✘ ✔ 𝑘 𝑖 𝑗 ℓ 1%
99%
Good compromise possible
All requirements seen so far forged into math on the left
Results shown on the right
Trade off between main and secondary objective on the right (in varying degrees)
Copyright © 2019 Timmy Schüller, Osnabrück University

13. Even More Real-World Requirements
Reconfigure rarely & little
Be resilient against failures
Do we need to
re-optimize?
Hardware outage
True changes of the topology (e.g. adding a new link)
Time of optimization or deployment of SR tunnels
Irregularities in traffic matrices (e.g. Superbowl)
These requirements can be seen as two sides of the same coin. If the optimization is resilient, naturally it will remain stable for longer periods of time.
Yet, this is a very hard challenge that I am working on right now
Copyright © 2019 Timmy Schüller, Osnabrück University

14. Any Questions or Comments?
The Takeaway
SR is a powerful Traffic Engineering tool
It has to be carefully fine- tuned towards specific use-case
Number of SR tunnels
Traffic splitting
Special nodes
Latency
etc.
Any Questions or Comments?
Time‘s up? Find me at
or contact me directly:
Etc = resiliency, add QOS classes
Details on this work can be found in: Schüller et al., “Traffic Engineering using Segment Routing and Considering Requirements of a Carrier IP Network”, IEEE/ACM Trans-actions on Networking, vol. 26, no. 4, pp , 2018
Copyright © 2019 Timmy Schüller, Osnabrück University