1. The Datapository Dave Andersen, CMU James Moss, CMU Nick Feamster, Georgia Tech http://www.datapository.net/

2. Overview Two aspects: –Network data repository –Supporting analysis tools and infrastructure Archive of network data –BGP: RouteViews, Abilene, RIPE –ISIS: Abilene –Spam data: Georgia Tech traps, etc. –NetFlow data: Abilene data –Wide-area probing data

3. Current Status Grabbing lots of data –Public: Abilene (BGP, IS-IS, Config, etc.), Routeviews (BGP updates and tables) –Semi-public: Georgia Tech campus monitoring probes (ping, traceroute, etc.), spam trap data, etc. –Brokered: Abilene NetFlow –Various other private data Mirroring at CMU and Georgia Tech –~ 12 TB at each site Mysql to postgres migration

4. Datapository Architecture Separate: collection, storage, analysis Collection: abstract type, format, and access method

5. Export: Formats and Applications Multiple ways to access data –Web Interface –XMLRPC Text-based output Programmatic interface Output to Matlab –Direct Postgres table access –Raw data Applications –Correlation across data streams –Longitudinal studies

6. Interaction: Web Interface

7. Example: Do Spammers Hijack BGP Routes? Theory: 1. announce BGP route for mail server 2. Send lots of spam 3. Withdraw route, becoming invisible Reality? Lets check...

8. Selecting Spammers SELECT * from spam WHERE spam.time >... AND spam.time spam.time - 20 AND bgp.time < spam.time + 3600 AND bgp.prefix = (spam.client_ip & ((~0) << (32 - bgp.mask)))