Presentation on theme: "Networks and Distributed Systems a.k.a. G22.3033-010G22.3033-010 Lakshmi Subramanian Jinyang Li"— Presentation transcript:
Networks and Distributed Systems a.k.a. G22.3033-010G22.3033-010 Lakshmi Subramanian http://cs.nyu.edu/~lakshmi Jinyang Li http://cs.nyu.edu/~jinyang
Class goals Help you –critically appreciate networks & systems research –learn creative problem solving (i.e. doing research) How? –Lectures/readings: discuss state-of-art work –Programming labs: play with real systems –A semester-long research project
Syllabus, grading etc. http://www.cs.nyu.edu/courses/fall06/G22.3033-010 Class participation (20%) –Read assigned papers before class! Two labs (10%) One project (70%) –Team of 2-3 people (<= 1 Ph.D. student per group) –Start next week –Weekly (or once every two weeks) meetings
Who should take the class? Grad-level class –Satisfy M.S. requirement of a “project” course Pre-requisite: –Basic knowledge on networks Computer Networks (L. Peterson) An engineering approach to computer networking (S. Keshav) –Programming experience TCP/IP Illustrated (R. Stevens)
Sensor-net challenges Different communication paradigm –host-to-host is the wrong fit –Data-centric Limited resources –Low radio bandwidth 250Kbps advertised, ~80Kbps in real life –Slow processor, tiny storage 8MHz CPU, 8K RAM –Limited energy
Overlays and P2P Distributed systems meet the Internet
Why p2p/overlay? A distributed system architecture: –No (minimal) centralized control –Nodes are symmetric in function Enabled by technology improvements Internet
Large scale wide-area systems Unmanaged (open p2p systems): – BitTorrent: >1M nodes – Skype: >5M users Managed –PlanetLab: 700 nodes over 336 sites –Akamai CDN: >10K nodes
Challenges How to find data? How to deal with failures? –Nodes fail and recover –Network outage and partition (Open networks only) How to deal with selfish or malicious nodes? –provide data integrity –provide privacy or anonymity
Challenge #1: resource discovery Case study: file sharing Where is the file named “Hamlet”?
Challenge #2: churn What if the node with “Hamlet” goes down?
Challenge #3: selfish nodes Selfish nodes do not want to upload “Hamlet” I do NOT have Hamlet
Challenge #4: malicious nodes I HAVE junk named Hamlet Malicious nodes lie about their contents
Check out the Spring class “distributed storage systems” Distributed systems in a data-center Connected by LANs low loss and delay Provide infrastructural services for apps –Network file systems –Databases –Distributed data processing