1 COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. Cognitive Security: Security Analytics and Autonomics for Virtualized Networks Lalita Jagadeesan (with Vijay Gurbani, Alan Mc Bride, Jie Yang) Bell Labs & CTO Security Group, Alcatel-Lucent Oct 6, 2015

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 2 THE DYNAMICS OF CLOUD & SOFTWARE DEFINED NETWORKS: OPPORTUNITIES AND THREATS Current state: Emerging network technologies are enabling applications to become portable, mobile, and borderless. Threats exploiting networks and applications are unpredictable and on the rise. Problem: Real-time prediction, detection and mitigation of security is lagging behind the fast paced migration of applications to the cloud environment Our approach: ­ Develop new algorithms and data analytics techniques to predict and detect known and unknown security threats. ­ Automate reconfiguration of virtualized security functionality for networks and applications Our goal ­ Enable networks to automatically detect security threats in real- time, dynamically reconfigure themselves to protect against these threats, and automatically immunize themselves against emerging and evolving threats

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 3 Cloud and software-defined networking brings new security challenges Emerging, evolving, and unknown threats on new kinds of virtualized networks Virtualized networks bring new opportunities Dynamically change security policy (e.g. firewall rules) Instantiate virtualized security functions closer to threats Dynamically migrate functionality to other virtual machines or other parts of the network when security issue detected Real-time machine-learning based streaming analytics + streaming anomaly detection Can help to proactively identify and detect unknown threats Limitations of current technologies (e.g., traditional SIEM) Signature based, can only address known threats Lack of flexibility, scalability, usability Require very labor intensive setup and tuning to be effective Motivation

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 4 ANALYTICS-DRIVEN DETECTION AND RESPONSE Analytics and Autonomics ­ Use machine learning to automatically detect anomalies ­ Normal behavior not fully known -- cannot accurately label past data and/or train machine learning algorithms on past normal behavior ­ Leverage dynamic capabilities of NFV and SDN networks for autonomic response (Distributed) Denial of Service ­ Distinguish abnormally high rates of legitimate traffic from malicious traffic ­ Legitimate traffic: input to cloud growth engine to instantiate new resources ­ Malicious traffic: input to cloud growth engine not to increase resources, security autonomics CAN UNSUPERVISED MACHINE LEARNING ON STREAMING DATA BE USED? Legitimate traffic -> cloud growth Malicious traffic -> security mitigations

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 5 ANOMALY DETECTION FOR SIP FLOODING GENERAL-PURPOSE UNSUPERVISED LEARNING TO IDENTIFY ANOMALIES No distinction between abnormally high rates of legitimate traffic and malicious traffic Abnormally high rates of legitimate traffic Malicious traffic (attack traffic does not send ACKs) Used a general- purpose anomaly detection application based on unsupervised machine learning for streaming data SIP = “Session Initiation Protocol”

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 6 OUR APPROACH: DOMAIN-SPECIFIC ALGORITHMS BASED ON TEMPORAL LOGIC & STATE MACHINES Normal sequence: [INVITE, 200OK, ACK] Aim to distinguish abnormally high rates of legitimate traffic from malicious traffic SIP protocol specifies a 3-way handshake: [client sends INVITE to server, server responds with 200 OK to client, client sends a matching ACK within 32 seconds] Open handshake: [INVITE, 200OK, time-out] can indicate malicious behavior (forces server to keep state waiting for ACK) Incorporate domain-specific knowledge: e.g. every 200 OK must be followed by a matching ACK within 32 seconds Invoke run-time verification algorithm when anomalies are detected by general-purpose anomaly detection – avoids run- time costs of running continually Learn blacklist (based on open handshakes) and incorporate into algorithm to provide information to security autonomics EXTEND ANOMALY DETECTION WITH DOMAIN-SPECIFIC ALGORITHMS High rates of time-outs can indicate distributed denial of service (DDoS) attack

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 7 PROOF-OF-CONCEPT ARCHITECTURE Commercial analytics platform with a machine learning application Our run-time verification algorithms built using the Python SDK of commercial platform Temporal logic/ state machine based properties monitored at run-time: e.g. “every 200 OK must be followed by a matching ACK within 32 seconds” SIP = “Session Initiation Protocol”, SIPp traffic generator

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 8 SCENARIO: LEGITIMATE AND MALICIOUS SIP TRAFFIC Period 1 (High rate of legitimate traffic) Two peaks of legitimate traffic Period 2 (Malicious traffic) Two peaks of malicious traffic Period 3 (Mixed traffic) One peak of each Peak traffic 30 msg/sec, baseline 10 msg/sec, addresses spoofed from a pool

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 9 GENERAL-PURPOSE ANOMALY DETECTION ON SCENARIO Does NOT correctly identify the three periods However, this anomaly detection application can be used as a trigger for our domain- specific algorithms

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 10 IDENTIFYING SUSPICIOUS TRAFFIC Our domain- specific algorithm identifies suspicious traffic based on open handshakes Suspicious calls detected after 32 seconds (timeout period)

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 11 MALICIOUS TRAFFIC AND BLACKLISTS (SIMULATION) Source addresses for malicious calls are placed on blacklist Suspicious calls blocked by blacklistSuspicious calls placed on blacklist

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 12 MALICIOUS TRAFFIC AND BLACKLIST FILTERING (FIREWALL) Suspicious calls are filtered by dynamically adding a new firewall rule PUTTING IT TOGETHER: ANALYTICS-DRIVEN SECURITY AUTONOMICS

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 13 INCREASING ATTACK TRAFFIC Increasing rates of attack can be detected through anomaly detection More significant attack drives more sophisticated security autonomics, e.g. instantiation of a new virtualized firewall LEVERAGING VIRTUALIZED NETWORK CAPABILITIES FOR SECURITY AUTONOMICS

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 14 Summary: ­ Analytics-driven security autonomics that leverages dynamic reconfiguration capabilities of virtualized networks ­ Analytics approach based on a combination of machine learning and run-time verification through domain-specific algorithms ­ Proof-of-concept architecture applied to SIP DDoS scenarios Future work ­ Extend machine learning algorithms and domain-specific knowledge to known and unknown threats on a broad range of protocols, and more fully integrate machine learning and run-time verification ­ Extend proof-of-concept architecture to include open-source analytics platforms such as Spark Streaming, and build upon Python machine learning libraries ­ Extend approach and proof-of-concept studies to include more sophisticated security autonomics CONCLUSIONS AND FUTURE WORK

COPYRIGHT © 2015 ALCATEL-LUCENT. ALL RIGHTS RESERVED. 15