1. 1 A Note on Emerging Science for Interdependent Networks Junshan Zhang School of ECEE, Arizona State University Network Science Workshop, July 2012 (Based on joint work with Osman Yagan and Dajun Qian)

2. 2 From Individual Networks to Network of Networks Networked systems: modern world consists of an intricate web of interconnected physical infrastructure and cyber systems, e.g., communication networks, power grid, transportation system, social networks, … Over the past few decades, there has been tremendous effort on studying individual networks: Communication networks, e.g., Internet, wireless, sensor nets, … Complex networks, e.g., E-R graph, small world model, scale-free networks … … Little attention has been paid to interdependent networks: Many networks have evolved to depend on each other, and depend heavily on cyber infrastructure in particular Focus of this talk: interdependent networks (e.g., cyber-physical systems)

3. Cyber-Physical Systems (CPS) A networked system consists of physical network and cyber network Emerging as the underpinning technology for 21th century Applications: smart grid, intelligent transportation system, manufacturing, etc. 3

4. 4 Interdependence: Operation of one network depends heavily on the functioning of the other network Q) what is the impact of interdependence between cyber-network and physical network? I) Vulnerability to cascading failures: node failures in one network may trigger a cascade of failures in both networks, and overall damage on interdependent networks can be catastrophic. II) Acceleration of information diffusion: conjoining can speed up information propagation in interdependent networks. CPS - Two Interdependent Networks

5. 5 Q) What is the impact of interdependence on cascading failures between cyber-network and physical network? Part I: Impact of Network Interdependence on Cascading Failures More susceptible to cascading failure due to interdependence How to design a system with better resilience against cascading failures?

6. 6 Network interdependence Power station operation relies on the control of nodes in cyber infrastructure Cyber nodes need power supply from power stations Vulnerability to cascading failures Even failures of a very small fraction of nodes may trigger a cascade of failures and result in a large scale blackout, e.g, blackout in Italy 2003 An Example: Modern Power Grid Power systems in Italy [Nature 2010]

7. Case Study on WTC Disaster 7 Telecommunication: e.g., Verizon lost 200K voice lines and 4.4M data circuits; 71% volume increase in 911 service and was switched to Brooklyn office Electric power system lost 3 substations, 5 distribution networks, … Q) Which parts are most vulnerable and which other parts are most resilient? Where are interdependences?

8. 8 2015-5-23 Network Model I Two interconnected networks need mutual support Initial setting: a fraction 1-p of A-nodes failed. Approach: To quantify ultimate functioning giant component size and critical threshold p Net A: Power grid Net B: Cyber infrastructure Inter-edge

9. 9 2015-5-23 Giant Connected Component (GCC) Model “one-to-one correspondence” [Nature 2010] Inter-edge: specify interdependence between two networks Assumption: a node can “function” only if belongs to the giant connected component of its own network has at least one inter-edge (support) from the other network Intra-edge: connections between nodes in same network Net. ANet. B

10. 10 2015-5-23 An Illustration of Cascading Failures attack step1 After a4 is removed, a3 fails since it is no longer in the giant component in A The intra & inter edges associated with a3 and a4 will be removed Step 1 Step 2 Step 3 Functioning giant component step2 b4 and b3 will be removed due to losing inter-edges from A step3 Cascading failure stops

11. 11 Allocation Strategies for Inter-Edges Q) How to allocate inter-edges against cascading failures? Random allocationOur strategy Number of inter-edges each node random; following binomial distribution Uniform: the same for all nodes Direction of inter-edge (support from nodes in the other network) Uni-directional: unilateral support from a node in the other network Bi-directional: mutual support between two connected nodes Critical threshold p c : minimum p that ensures the existence of functioning giant component after cascading failures; higher p c means less tolerant to network failures (lower robustness) and vice versa Metric for robustness:

12. 12 Analysis of Cascading Failures Notation: p Ai ; p Bi : the functioning giant component fraction in A (resp. B) at stage i p’ Ai ; p’ Bi : the remaining fraction in A (or B) which are equivalent to node removals due to cascading failures up to stage i P A (p); P B (p): After a fraction 1-p of A-nodes (B-nodes) failed, the giant component fraction out of remaining pN nodes

13. 13 Analysis of Cascading Failures Uniform Allocation of Bi-directional Inter-Edges Stage 1: Node failures in Network A inter-edge can be disconnected w.p. 1-p A1 The remaining fraction of nodes with inter-edges: p’ B2 = 1-(1-p A1 ) k Random failures of 1-p of nodes Removal of inter-edges functioning giant component A 1 p A1 =pP A (p) Stage 2: Cascading effect of A-node failures on network B functioning giant component B 2 p B2 =p’ B2 P B (p’ B2 ) Notation: P A (p), P B (p): After a fraction 1-p of A-nodes (B-nodes) failed, the giant component fraction out of remaining pN nodes

14. 14 Stage 3: Network A ’s further fragmentation due to B-node failures inter-edge can be disconnected w.p. 1-P B ( p ’ B2 ) The remaining fraction of A 1 : 1-(1-P B (p’ B2 )) k For A, the joint effect of Stage 1 & 3 on A amount to node failures in A with fraction 1-p’ A3 =1- p+p(1-P B (p’ B 2 )) k Key step: further node failures in A 1 at Stage 3 has the same effect as taking out equivalent fraction of nodes in A functioning giant component A3 p A3 =p’ A3 P A (p’ A3 ) Uniform Allocation of B-directional Edges (Cont’d)

15. network A network B p A1 =pP A (p) p B2 =p’ B 2 P B (p’ B2 ) p A3 =p’ A3 P A (p’ A3 ) p B4 =p’ B2 P A (p’ B2 ) …. 15 The recursive process reaches stead state By calculating the equilibrium point, we can get the ultimate giant component size and critical threshold functioning giant component size in dynamics of cascading failures Stage 1 Stage 3 Stage 2 Stage 4 Uniform Allocation of Bi-directional Edges (Cont’d)

16. 16 Uniform vs. Random Allocation Observation: Uniform allocation leads to higher robustness than random allocation Intuition: Random allocation can result in a non-negligible fraction of nodes with no inter-network support, whereas uniform allocation can guarantee support for all nodes uniform allocation Random allocation No support

17. 17 Uni-directional v.s. Bi-directional Observation The bi-directional inter-edges can better combat the cascading failures than uni- directional inter-edges The cascading failures are more likely to spread with uni-directional edges For fair comparison, the total number of uni-directional edges should be twice the number of bi-directional edges

18. 18 2015-5-23 Numerical Example Two Erdos-Renyi networks with average intra-degree fixed at 4 The p c varies over different average inter-degree k As expected, the uniform & bi-directional allocation leads to the lowest p c under various conditions Lower p c indicates the higher robustness

19. 19 Limitation of GCC Model for Physical Network Giant Connected Component (GCC) model [Nature 2010] Assumption: Only the nodes in the largest connected component can work properly Pros: facilitate theoretical analysis Cons: Cannot capture some key features of physical network, e.g., power grid

20. 20 2015-5-23 Shortcoming of GCC Model for Power Grid

21. Threshold model [Gleeson 07] A node would fail if the fraction of its failed neighbors exceeds the threshold; capture the load redistribution feature 21 2015-5-23 Threshold Model the more power stations fail, the more load being redistributed to A A: more likely to fail

22. 22 2015-5-23 Network Model II Two interdependent networks with mutual support -GCC-model for cyber-network; -threshold model for physical infrastructure Power grid Cyber-network

23. 23 2015-5-23 GCC-model  All power stations cannot function in subcritical region GCC Model vs Threshold Model power grid  micro-grids: isolated power stations can still function Defensive Islanding: islanding can prevent further failure spreading Sparsely connected regime (low average degree) Threshold model  isolated components can still function  the propagation of cascading failure is constrained by isolated components

24. 24 GCC Model vs Threshold Model Threshold model  A small fraction of node failures may lead to network collapse Large scale blackout can be triggered by one station failure, e.g., Italy black out 2003 power grid GCC-model  cascading failures cannot happen if initially failed fraction q is small Densely connected regime (high average degree) Main points:  GCC model underestimates the damages that could be triggered by a small fraction of node failures  Threshold model captures some key features of power grid

25. 25 Robustness performance (initial failed fraction q=0.1%) small initial failures that have negligible impact on single physical network may damage overall CPS (with high degree and low threshold) Robustness of CPS model II

26. 26 2015-5-23 Robustness performance (when initial failure q=0.1%) A single network (high link degree and low threshold) may be resistant to a small fraction of node failures. However, such failures can still be disastrous in interconnected networks low zmedium to high zVery high z high thresholdresulting in a small fraction of further node failures low threshold partial breakdown total breakdown (in contrast to single network) Further Results in Interdependent Networks

27. 27 2015-5-23 Single network (low threshold) - Each node can tolerate more neighbors’ failures - Very few node failures are difficult to incur further failures; although still susceptible to large initial failures Interdependent networks (low threshold) -the scale of node failures can be “amplified” due to cascading failures between two networks -the system is vulnerable to a small fraction of node failures Densely Connected Regime Intuition:

28. 28 2015-5-23 Information cascade Part II: Impact of Network Interdependence on Information Diffusion information epidemic real-time information propagation interdependence between two networks can facilitate information diffusion Q) What is the impact of interdependence on information diffusion in overlaying social-physical networks?

29. 29 “A social network is a social structure made up of a set of actors (e.g., individuals or organizations) and the dyadic ties between these actors (e.g., relationships, connections, or interactions)” [Wiki] Social-Physical Networks Online social networkPhysical information network -Traditional “physical” interactions: e.g., face-to-face contacts, phone calls … Social-physical network: medium for information diffusion

30. 30 “Multi-member’’ Individuals can be member of multiple social networks Interdependence across Multiple Networks “coupling’’ Different social networks can “overlap” due to “multi- member” individuals Q): How does information propagate across multiple interdependent networks?

31. 31 Model: Overlaying Social-Physical Networks W : physical info network F : online social network  n nodes in physical information network; only one online social network  Each individual in W participates in F with probability α  Each node in W has neighbors with  Each node in F has online neighbors with online connection physical interactions online membership individual same person

32. 32 Information Cascade information diffusion in one network can trigger the propagation in another network and may help information diffusion interdependence between multiple networks online social network physical info network

33. 33 SIR Model for Information Diffusion  Message can successfully spread along a link that corresponds to physical interaction or online communication with probabilities and, respectively Only existing links can be used in spreading the information

34. 34 “Giant component”: the largest connected component in the network Questions  When an information epidemic can take place?  What is the size of information epidemic?  When a giant component that occupies a positive fraction of nodes can appear?  What is the fractional size of giant component? Information Cascade in Overlaying Social-Physical Networks

35. 35 Challenge  How to characterize the phase transition behavior (existence condition and size of giant component) in two overlaying graphs? Key idea  Treat the overlaying networks as an inhomogeneous random graph Approaches  Colored degree-driven random graphs with different types of links [Soderberg 2003] general case: nodes in F and W have arbitrary degree distributions  Inhomogeneous random graph with different types of nodes [Bollobás et al. 2007] Alternative approach for a special case where nodes in F and W have Poisson degree distributions, i.e., F and W are Erdős–Rényi graphs Analysis of Information Diffusion

36. 36 General Case: Graphs with Arbitrary Degree Distributions Original overlaying networks can be modeled as a random graph where nodes are connected by two types of links (online communications and physical interactions). The phase transition behaviors of the equivalent random graph can be characterized by capitalizing on mean-field approach [Soderberg 2003]. random graph with 2 types of links treat as a single node overlaying social-physical networks F W

37. 37 where Main Result I If the critical threshold, then with high probability there exists a giant component with size ; otherwise then the largest component has size The existence of the giant component is determined by the critical threshold  The critical threshold marks the “tipping point ” of information epidemics.

38. 38 The fractional size of giant component in the random graph is given by Main result II where h 1 and h 2 in (0,1] are given by the smallest solution of  The fractional size of giant component gives the fractional size of individuals that receive the message.

39. 39 αrequirement for the existence of giant component when 0.1 0.5 0.9 overlaying social-physical networks single network [Newman 2002]  If the network W and F are disjoint, an information epidemic can occur only if or Main point:  Two networks, although having no giant component individually, can yield an information epidemic when they are conjoined together Numerical Result: Critical Threshold

40. 40 Special Case: Erdős–Rényi Graph  graph W has n nodes; each node in W participates F w.p. α  any two nodes in W are connected w.p.  any two nodes in F are connected w.p. Scenario: overlaying Erdős–Rényi Graphs Approach: inhomogeneous random graph [ Bollobás 2007]  can quantify the size of the second largest connected component when a giant component exists  gives a tighter bound on the largest connected component when a giant component does not exist

41. 41  Critical threshold: If, then w.h.p. the largest component has size and the second largest component has size. If, then the largest component has size.  Fractional size of giant component: Special Case: Erdős–Rényi Graph where ρ1 and ρ2 in [0,1] are determined by the largest solution to

42. 42 2015-5-23 Impact of Network Interdependence on Information Diffusion We focus on information diffusion in an overlaying social- physical network, where message spreads amongst people through both physical interactions and online communications. We show that even if there is no information epidemic in individual networks, information epidemics can take place in the conjoint social-physical network We show that the critical threshold and the size of information epidemics can be precisely determined using inhomogeneous random graph models.

43. Phase Transition Behavior Information Diffusion vs. Cascading Failures 43 Information Diffusion Cascading Failures

44. Information Diffusion - v_1, v_9, v_10 are not Facebook users - Information starts at node v_1 Giant Component of W consists of {v_1,v_2,v_3,v_5,v_6,v_7,v_9 } Giant Component of F consists of {v_2,v_3,v_4,v_5,v_6,v_7,v_8 } Giant Component of FUW consists of {v_1, …, v_10}  nodes that receive the information Information does cascade between the two networks, but the eventual cascade size can be computed by the giant component size of the conjoint network H = F U W.  Behavior boils down to the phase transition of a single combined network.  Second-order (continuous) phase transition W F Initial set-up W F Propagation to 1 st hop neighbors W F Propagation to 2 st hop neighbors W F Propagation to 3 st hop neighbors W F Steady state

45. Cascading Failures - Net A and Net B are defined on disjoint vertex sets. - Initially node v_1 fails. Giant Component of A consists of {v_1,v_2,v_3,v_5,v_6,v_7,v_9 } Giant Component of B consists of {v_2,v_3,v_4,v_5,v_6,v_7,v_8 } At each stage, only the Giant Component of the functional nodes remain.  A giant component computation is required at each stage While failures cascade between the two networks, the network reduces to its giant component at each step.  the overall dynamics is equivalent to the superimposition of possibly many phase transitions.  First-order (discontinuous) phase transition A B Net A: Only the giant component survives Net B: Only nodes that have support survive Net B: Only the giant component survives Net A: Only nodes that have support survive

46. 46 Conclusions We investigate the impact of interdependence between cyber- network and physical network: I) Vulnerability to cascading failures: node failures in one network may trigger a cascade of failures in both networks. To improve the robustness of interdependent networks, we proposed some strategy for allocating inter-edges. II) Acceleration of information diffusion: conjoining can speed up information propagation in coupled networks. There are still many open questions on network interdependence. Need a foundation for interdependent networks!