Presentation is loading. Please wait.

Presentation is loading. Please wait.

The Datapository Dave Andersen, CMU James Moss, CMU Nick Feamster, Georgia Tech

Similar presentations


Presentation on theme: "The Datapository Dave Andersen, CMU James Moss, CMU Nick Feamster, Georgia Tech"— Presentation transcript:

1 The Datapository Dave Andersen, CMU James Moss, CMU Nick Feamster, Georgia Tech

2 Overview Two aspects: –Network data repository –Supporting analysis tools and infrastructure Archive of network data –BGP: RouteViews, Abilene, RIPE –ISIS: Abilene –Spam data: Georgia Tech traps, etc. –NetFlow data: Abilene data –Wide-area probing data

3 Current Status Grabbing lots of data –Public: Abilene (BGP, IS-IS, Config, etc.), Routeviews (BGP updates and tables) –Semi-public: Georgia Tech campus monitoring probes (ping, traceroute, etc.), spam trap data, etc. –Brokered: Abilene NetFlow –Various other private data Mirroring at CMU and Georgia Tech –~ 12 TB at each site Mysql to postgres migration

4 Datapository Architecture Separate: collection, storage, analysis Collection: abstract type, format, and access method

5 Export: Formats and Applications Multiple ways to access data –Web Interface –XMLRPC Text-based output Programmatic interface Output to Matlab –Direct Postgres table access –Raw data Applications –Correlation across data streams –Longitudinal studies

6 Interaction: Web Interface

7 Example: Do Spammers Hijack BGP Routes? Theory: 1. announce BGP route for mail server 2. Send lots of spam 3. Withdraw route, becoming invisible Reality? Lets check...

8 Selecting Spammers SELECT * from spam WHERE spam.time >... AND spam.time spam.time - 20 AND bgp.time < spam.time AND bgp.prefix = (spam.client_ip & ((~0) << (32 - bgp.mask)))


Download ppt "The Datapository Dave Andersen, CMU James Moss, CMU Nick Feamster, Georgia Tech"

Similar presentations


Ads by Google