1. Algorithm: For all e E t, define X e = {w e if e G t, 1 - w e otherwise}. Measure likelihood of substructure S by. Flag S as anomalous if, where is an anomalicity threshold. Substructures are then analyzed in decreasing order of anomalicity as resources and time allow. DAPA-V10 is efficient: run-time complexity is O(|E T |). A persistent pattern is a collection of vertices that (1) form a connected component. (2) communicate regularly. Given a volatile time-evolving network: (1) Find persistent patterns. (2) Detect local and global anomalous activity. Challenges: Volatility: Network changes drastically, frequently. Sparsity: A single snapshot is extremely sparse. Scalability: Algorithms must be efficient for large networks of potentially millions of members. DAPA-V10: Discovery and Analysis of Patterns and Anomalies in Volatile Time-Evolving Networks Problem Statement Persistent Patterns We discover persistent patterns in volatile time-evolving networks and use them to find and rank anomalous events. Previous work focuses on identifying times of higher activity overall. DAPA-V10 detects local anomalies, pinpointing sources of unusual behavior for further analysis. Our approach is scalable to very large networks. Our Algorithm: DAPA-V10Experimental Results Conclusions Dataset: a collection of email correspondence between 672 Enron employees from 1997-2002. Found 6 persistent patterns that represent connected components of employees with regular communication. Substructures of edges within and between persistent patterns are monitored over time for anomalous behavior. Anomalies found by DAPA-V10 correspond with events surrounding the Enron scandal. The close correspondence illustrates the effectiveness of our approach. Brian Thompson Rutgers University bthom@cs.rutgers.edu Tina Eliassi-Rad Lawrence Livermore Lab eliassirad1@llnl.gov Anomaly Detection Network Representation Model a network as a dynamic graph G=(V,E T ). To capture temporal information, we construct a weighted cumulative graph G’=(V,E t ’). Edge weights are defined by a decay function f: SourceDest.t_startt_end v49273v71192t = 5t = 9 v83492v12987t = 12t = 14 v40927v62198t = 13t = 16 v98364v39872t = 20t = 21 v18964v38719t = 20t = 25 1. Timestamped edges are used to construct a dynamic graph. 2. A cumulative graph is used to measure the average strengths of relationships. 3. Persistent patterns are identi- fied. Substructures are selected to track activity both within and between components. 4. Substructures are monitored, flagging abnormal activity for investigation and analysis. Algorithm: Consider only edges with weight above threshold θ. Decrease θ until a component of size appears. Remove edges and iterate on the remaining graph. Goal: identify anomalies on a local and global scale. Monitor substructures: sets of edges (1) within each persistent pattern, and (2) between each pair of patterns. A substructure is anomalous if recent activity across its edges differs significantly from what is expected. Timeline of Enron Scandal TimeEvent 2/01Executives get $1M bonuses; stock is soaring 4/01Q1 profit $536M; Wall St. analyst suspicious 7/01Reported earnings $50B; share price dropping 8/01Public criticism of Enron accounting practices 9/019/11 attacks; Enron director sells 500K shares 10/01Q3 loss of $618M; SEC begins investigation 11/01Acquisition offer, revoked; ‘junk’ credit rating 12/01Enron files for bankruptcy, lays off employees Distribution of edge weights and threshold points for Enron data. Resulting persistent patterns shown in figure at center. Future work: Conduct experiments on a variety of domains (e.g. cyber). Use Enron dataset to evaluate effectiveness of DAPA-V10 as an early predictor of high-impact events. Normalize at each time step to find local anomalies independent of global trends in network activity. Incorporate semantic information from complex networks.