Aaron Gember-Jacobson, Chaithan Prakash, Raajay Viswanathan, Robert Grandl, Junaid Khalid, Sourav Das, Aditya Akella 1 OpenNF: Enabling Innovation in Network Function Control Presenter: Xin Li  These slides are modified from the original slides

Network functions (NFs) Perform sophisticated stateful actions on packets/flows 2 Intrusion detection system (IDS) Caching proxy WAN optimizer

NF trends NFV → dynamically allocate NF instances SDN → dynamically reroute flows Dynamic reallocation of packet processing 3 Intrusion detection system (IDS) Caching proxy WAN optimizer Xen/KVM

Virtual machine replication – Unneeded state → incorrect actions – Cannot combine → limited reallocation Split/Merge [NSDI’13] – State allocations and accesses occur via library – Addresses a specific problem → limited suitability – Packets may be dropped or re-ordered → wrong NF behavior 4 Existing approaches

Example: elastic NF scaling 1.Satisfy performance SLAs 2.Minimize operating costs 3.Accurately monitor traffic 5 CPU Packet loss

Example: elastic NF scaling 1.Satisfy performance SLAs 2.Minimize operating costs 3.Accurately monitor traffic 6 CPU Packet loss To simultaneously… Problem: NFV+SDN is insufficient Cannot effectively implement new services or abstractions!

Why NFV + SDN falls short 1. SLAs2. Cost3. Accuracy Reroute new flows [Stratos - arXiv: ] Reroute existing flows [SIMPLE - SIGCOMM ‘13] Wait for flows to die [Stratos - arXiv: ] 7 ? Packet loss SLA: <1%

SLAs + cost + accuracy: What do we need? Quickly move, copy, or share internal NF state alongside updates to network forwarding state Guarantees: loss-free, order-preserving, … 8 … 123 … Also applies to other scenarios

Outline Motivation and requirements Challenges OpenNF architecture – State export/import – State operations – Guarantees Evaluation 9

1.Supporting many NFs with minimal changes 2.Dealing with race conditions 3.Bounding overhead Challenges 10 State Packet Route Update

OpenNF overview 11 NF State Manager Flow Manager OpenNF Controller Control Application move/copy/share state export/import State

State created or updated by an NF applies to either a single flow or a collection of flows NF state taxonomy 12 Connection TcpAnalyzer HttpAnalyzer TcpAnalyzer HttpAnalyzer Per-flow state ConnCount Multi-flow state All-flows state Statistics

NF API: export/import state Functions: get, put, delete 13 No need to expose/change internal state organization! Filter Per Multi All Scope NF get put

Control operations: move 14 NF State Manager Control Application move (port=80, Bro 1, Bro 2 ) get(per, port=80) [Chunk1] put (per, Chunk1) del(per, port=80) [Chunk2] put (per, Chunk2) forward(port=80, Bro 2 ) Flow Manager Bro 2 Bro 1 Also provide copy and share

detect- MHR Split/Merge [NSDI ‘13] : pause traffic, buffer packets – Packets in-transit when buffering starts are dropped Lost updates during move 15 B1 R1 R2 Missing state Bro 2 Bro 1 move(red,Bro 1,Bro 2 ) Missing updates Loss-free: All state updates should be reflected in the transferred state, and all packets should be processed R3

NF API: observe/prevent updates using events 16 Only need to change an NF’s receive packet function! R1 R2 B1 R1 NF

1. enableEvents(red,drop) on Bro 1 2. get / delete on Bro 1 3.Buffer events at controller 4. put on Bro 2 5.Flush packets in events to Bro 2 6.Update forwarding Use events for loss-free move 17 Bro 2 Bro 1 R3R1 Drop R1 R1,R2 R2 R1,R2,R3

False positives from Bro’s weird script Re-ordering of packets 18 Order-preserving: All packets should be processed in the order they were forwarded by the switch Order-preserving: All packets should be processed in the order they were forwarded by the switch Controller Switch Bro 2 5. Flush buffer 6. Request forwarding update Bro 1 R2 R4 R3 R2 R4 R3

Flush packets in events to Inst 2 enableEvents(blue,buffer) on Inst 2 Forwarding update: send to Inst 1 & controller Wait for packet from switch (remember last) Forwarding update: send to Inst 2 Wait for event for last packet from Inst 2 Release buffer of packets on Inst 2 Order-preserving move 19 B1 Drop B1B1,B2 B2 B1,B2, B3 Buf B3 B4 B1,B2, B3,B4

Copy and share operations Used when multiple instances need some state Copy – no or eventual consistency – Once, periodically, based on events, etc. Share – strong or strict consistency – Events are raised for all packets – Events are released one at a time – State is copied before releasing the next event 20 Copy (multi-flow): 111ms Share (strong): 13ms/packet Copy (multi-flow): 111ms Share (strong): 13ms/packet

Applications decide (based on NF & objectives): 1.Granularity of operations 2.Guarantees desired Bounding overhead 21 Filter Per Multi All Scope … LF LF+OP 123 … … + None

1.Dealing with diversity 2.Dealing with race conditions OpenNF: SLAs + cost + accuracy 22 Export/import state based on its association with flows Events Lock-step forwarding updates +

Implementation Controller (3.8K lines of Java) Communication library (2.6K lines of C) Modified NFs (3-8% increase in code) 23 Bro IDS iptables Squid CachePRADS

Evaluation OpenFLow enabled HP ProCurve 6600 switch 4 mid-range servers – OpenNF controller – NFs – Traffic generators Two real traces and synethetic workloads 24

Example app: elastic NF scaling movePrefix(prefix,oldInst,newInst): copy(oldInst,newInst,{nw_src:prefix},multi) move(oldInst,newInst,{nw_src:prefix},per,LF+OP) while (true): sleep(60) copy(oldInst,newInst,{nw_src:prefix},multi) copy(newInst,oldInst,{nw_src:prefix},multi) scan.bro vulnerable.bro weird.bro scan.bro vulnerable.bro weird.bro 25

Overall benefits for elastic scaling Bro IDS processing 10K pkts/sec – At 180 sec: move HTTP flows (489) to new IDS – At 360 sec: move back to old IDS SLAs: 260ms to move (loss-free) Accuracy: same log entries as using one IDS – VM replication: incorrect log entries Cost: scale down after state is moved – Stratos: scale down delayed 25+ minutes 26 [arXiv: ]

Evaluation: importance of guarantees Bro 1 processing malicious 1K pkts/sec After 14K packets: move active flows to Bro 2 AlertBaselineNFLFLF+OP Incorrect file type MHR Match MD Total

PRADS asset detector processing 5K pkts/sec Move per-flow state for 500 flows Evaluation: operations 28 Packets dropped! packets in events Operations are efficient, but guarantees come at a cost! Operations are efficient, but guarantees come at a cost! 1120 pkts buffered 838 pkts in events + Bro: 5% of alerts missed! NG NG PL LF PL+ER OP PL+ER

Evaluation: impact of packet rate and per-flow states 29 Linear

Evaluation: state export/import 30 Serialization/deserialization costs dominate Cost grows with state complexity

PRADS asset detector processing 5K pkts/sec Move per-flow state for 500 flows Evaluation: operations 31 Packets dropped! packets in events Operations are efficient, but guarantees come at a cost! Operations are efficient, but guarantees come at a cost! 1120 pkts buffered 838 pkts in events + Bro: 5% of alerts missed! NG NG PL LF PL+ER OP PL+ER

Evaluation: impact of granular control Two squid instance Copy (after 20 sec) 32 No impact for first squid instance No state on the 2 nd squid instance Modest increase in hit ratio, but reduce traffic dramatically

Evaluation: controller scalability Improve scalability with P2P state transfers 33

Dynamic reallocation of packet processing enables new services Realizing SLAs + cost + accuracy requires quick, safe control of internal NF state OpenNF provides flexible and efficient control with few NF modifications Conclusion 34

Thanks! Question & Answer 35