Access Networks: Applications and Policy Nick Feamster CS 6250 Fall 2011 (HomeOS slides from Ratul Mahajan)

Huge amount of tech in homes

Home users struggle Management Nightmare Integration Hurdles

Why developers are not helping Application Hardware The actual devices in the house Application Topology Handle WiFi vs. 3G vs. Eth, Subnets Hardware The actual devices in the house Application Device Handle different brands/models Topology Handle WiFi vs. 3G vs. Eth, Subnets Hardware The actual devices in the house Application Coordination When apps disagree, who wins? Device Handle different brands/models Topology Handle WiFi vs. 3G vs. Eth, Subnets Hardware The actual devices in the house Application User Preference What is automated? When? How? Coordination When apps disagree, who wins? Device Handle different brands/models Topology Handle WiFi vs. 3G vs. Eth, Subnets Hardware The actual devices in the house Application Logic User Preference What is automated? When? How? Coordination When apps disagree, who wins? Device Handle different brands/models Topology Handle WiFi vs. 3G vs. Eth, Subnets Hardware The actual devices in the house

Vendors only build islands Vertically integrate hardware and software Seldom make use of other vendors devices No single vendor comes close to providing all the devices a home needs

Climate Control Remote Lock Camera- Based Entry Video Recording Interoperability is not sufficient Media: DLNA, AirTunes, etc. Devices: UPnP, SpeakEasy, mDNS, etc. Home Auto: Zwave ZigBee, X10, etc.

Monolithic systems are inextensible Security: ADT, Brinks, etc. Academic: EasyLiving, House_n, etc. Commercial: Control4, Elk M1, Leviton, etc. Home Media Security

An alternative approach: A home-wide operating system Operating System Video Rec. Remote Unlock Climate HomeStore

Goals of HomeOS Simplify application development Enable innovation and device differentiation Simplify user management

Simplify development … … App A App B Application Logic User Preference What is automated? When? How? Coordination When apps disagree, who wins? Device Handle different brands/models Topology Handle WiFi vs. 3G vs. Eth, Subnets Hardware The actual devices in the house

Application Logic User Preference What is automated? When? How? Coordination When apps disagree, who wins? Device Handle different brands/models Topology Handle WiFi vs. 3G vs. Eth, Subnets Hardware The actual devices in the house Application Logic User Preference What is automated? When? How? Coordination When apps disagree, who wins? Device Handle different brands/models Topology Logically centralize devices Hardware The actual devices in the house Application Logic User Preference What is automated? When? How? Coordination When apps disagree, who wins? Device Standardize at functional layer Topology Logically centralize devices Hardware The actual devices in the house Application Logic User Preference What is automated? When? How? Coordination Access control mediates conflicts Device Standardize at functional layer Topology Logically centralize devices Hardware The actual devices in the house Application Logic User Preference Users manage access control rules Coordination Access control mediates conflicts Device Standardize at functional layer Topology Logically centralize devices Hardware The actual devices in the house Simplify development … … App A App B Driver … Port … Access Control Mg mt UI Mg mt UI

Roles in HomeOS Roles are functional descriptions of ports –lightswitch, television, display, speakers, etc. –App developers program against roles Enable vendors to innovate/differentiate –Anyone can create a new role e.g., SonyBraviaTV vs. television Allows new functionality to be rapidly exposed –Commodity vendors can still participate

Simplify user management Conducted a field study –Modern homes with automation & other tech –14 homes, 31 people Users needs for access control –Applications as security principals –Time in access control decisions –Confidence in their configuration

Management primitives Datalog access control rules –(port, group, module, time-start, time-end, day, priority, access-mode) –Reliable reverse perspectives help users confidently configure access control User accounts –Can be restricted by time (guests) Application manifests –Specify role requirements for compatibility testing –Simplifies rule setup (only when roles match)

Implementation status Built on the.NET CLR ~15,000 lines of C# –~2,500 kernel 11 Applications –Average ~300 lines/app Music Follows the Lights –Play, pause & transfer music where lights are on/off Two-factor Authentication –Based on spoken password and face recognition

Open questions/Ongoing work Additional evaluation –Is it easy to write apps and drivers? –Is it easy to manage? –Does it scale to large homes? Deploy & support application development Explore business/economic issues

Summary A home-wide OS can make home technology manageable and programmable HomeOS balances stakeholder desires –Developers: abstracts four sources of heterogeneity –Vendors: enables innovation and differentiation –Users: provides mgmt. primitives match mental models

Detecting Network Neutrality Violations with Causal Inference Mukarram Bin Tariq, Murtaza Motiwala Nick Feamster, Mostafa Ammar Georgia Tech

19 November 6, 2006 The Network Neutrality Debate Users have little choice of access networks. ISPs want to share from monetizable traffic that they carry for content providers.

20 Goal: Make ISP Behavior Transparent Our goal: Transparency. Expose performance discrimination to users. Source: Glasnost project

21 Existing Techniques are Too Specific Detect specific discrimination methods and policies –Testing for TCP RST packets (Glasnost) –ToS-bits based de-prioritization (NetPolice) Limitations –Brittle: discrimination methods may evolve –Evadable ISP can whitelist certain servers, destinations, etc. ISP can prioritize monitoring probes Active probes may not reflect user performance Monitoring is not continuous

22 Main Idea: Detect Discrimination From Passively Collected Data Objective: Establish whether observed degradation in performance is caused by ISP Method: Passively collect performance data and analyze the extent to which an ISP causes this degradation This talk: Design, implementation, evaluation, and deployment of NANO

23 Ideal: Directly Estimate Causal Effect Baseline Performance Performance with the ISP Causal Effect = E(Real Throughput using ISP) E(Real Throughput not using ISP) Ground truth values for performance with and without the ISP (treatment variable) Problem: Need both ground truth values observed for same client. These values are typically not available.

24 Association = E(Observed Throughput using ISP) E ( Observed Throughput not using ISP) Instead: Estimate Association from Observed Data Observed Baseline Performance Observed Performance with the ISP Problem: Association does not equal causal effect. How to estimate causal effect from association?

25 Association is Not Causal Effect Comcast Other ISPs Avg. BitTorrent Throughput 5 kbps 10 kbps Comcast BT Throughput ? Client Setup TimeofDay ContentLocation Why? Confounding variables can confuse inference. Suppose Comcast users observe lower BitTorrent throughput. Can we assume that Comcast is discriminating? No! Other factors (confounders) may correlate with both the choice of ISP and the output variable.

26 Strawman: Random Treatment Treat subjects randomly, irrespective of their initial health. Measure association with new outcome. Association converges to causal effect if the confounding variables do not change during treatment. = = 0.55 Treated HHH HS Untreated H SS S S H H H SS S SS α θ Common approach in epidemiology. S = sick H = healthy

27 The Internet Does Not Permit Random Treatment Random treatment requires changing ISP. Problems –Cumbersome: Nearly impossible to achieve for large number of users –Does not eliminate all confounding variables (e.g., change of equipment at users home network) Alternate approach: Stratification

28 Stratification: Adjusting for Confounders Step 1: Enumerate confounders e.g., setup ={, } Step 2: Stratify along confounder variable values and measure association Association implies causation (no other explanation) HH H HH H HH H SS S H S S SS HH H H S S S S S HH HHH SS SS Treated Baseline Strata Causal Effect ( θ )

29 Stratification on the Internet: Challenges What is baseline performance? What are the confounding variables? Which data to use, and how to collect it? How to infer the discrimination method?

30 What is the baseline performance? Baseline: Service performance when ISP not used –Need some ISP for comparison Approach: Average performance over other ISPs Limitation: Other ISPs may also discriminate

31 What are the confounding variables? Client-side –Client setup: Network Setup, ISP contract –Application: Browser, BT Client, VoIP client –Resources: Memory, CPU, network utilization –Other: Location, number of users sharing home connection Temporal –Diurnal cycles, transient failures

32 What data to use; how to collect it? NANO-Agent: Client-side, passive collection –per-flow statistics: throughput, jitter, loss, RST packets –application associated with flow –resource monitoring CPU, memory, network utilization Performance statistics sent to NANO-Server –Monitoring, stratification, inference

33 Evaluation: Three Experiments Experiment 1: Simple Discrimination –HTTP Web service –Discriminating ISPs drop packets Experiment 2: Long Flow Discrimination –Two HTTP servers S 1 and S 2 –Discriminating ISPs throttle traffic for S1 or S2 if the transfer exceeds certain threshold Experiment 3: BitTorrent Discrimination –Discriminating ISP maintains list of preferred peers –Higher drop rate for BitTorrent traffic to non-preferred peers

34 Experiment Setup Access ISP 5 ISPs in Emulab 2 Discriminating Service Providers PlanetLab nodes HTTP and BitTorrent Discrimination Throttling and dropping Policy with Click router Confounding Variables Server location near servers (West coast nodes) far servers (remaining PlanetLab nodes) Internet D1D2N1N2N3 ~200 PlanetLab nodes ISPs Clients Running NANO-Agent

35 Without Stratification, Detecting Discrimination is Difficult Overall throughput distribution in discriminating and non-discriminating ISPs is similar. Simple Discrimination

36 Stratification Identifies Discrimination Discriminating ISPs have clearly identifiable causal effect on throughput Neutral ISPs are absolved SimpleLong-FlowBitTorrent

37 Implementation and Deployment Implementation –Linux version available –Windows and MacOS versions in progress Now: 27 users –Need thousands for inference Performance dashboard may help attract users Throughput DNS Latency Traffic Breakdown Performance Relative to Other Users

38 Summary and Next Steps Internet Service Providers discriminate against classes of users and application traffic today. Need passive approach –ISP discrimination techniques can evolve, or may not be known to users. –Tradeoff: Must be able to enumerate confounders NANO: Network Access Neutrality Observatory –Infers discrimination from passively collected data –Detection succeeds in controlled environments –Deployment in progress. Need more users.

39

40 NANO Can Infer Discrimination Criteria ISP throttles throughput of a flow larger than 13MB or about 10K packets cum_pkts not_discriminated cum_pkts > > discriminated EvaluationApproach

41 Sufficiency of Confounding Variables

42 Why Association != Causal Effect? Positive correlation in health and treatment Can we say that Aspirin causes better health? Confounding Variables correlate with both cause and outcome variables and confuse the causal inference Aspirin No Aspirin Healthy 40%15% Not Healthy 10%35% Aspirin Health ? Sleep Diet Other Drugs Age

43 Network Neutrality ISPs remain neutral in forwarding traffic irrespective of –Content: voice, video, data –Application: p2p, VoIP, VoD –Participants: Service providers, Google, Hulu, Youtube Discrimination: biased or non-neutral forwarding of traffic Focus of this paper –Detecting and Quantifying Discrimination

44 Inferring the Criteria Label data in two classes: –discriminated (-) –non-discriminated (+) Train a decision tree for classification –Rules provide hints about the criteria Criteria: youtube traffic, greater than 1 MB is affected

45 Discrimination can take many forms Blocking ports Disrupting connections, e.g., using TCP RST Throttling and prioritizing based on destination or service –Target domains, applications, or content Discriminatory peering –Resist peering with certain content providers

46 Causality: An Analogy from Health Epidemiology: study causal relationships between risk factors and health outcome NANO: infer causal relationship between ISP and service performance degradation

47 Without Stratification, Detecting Discrimination is Hard Overall throughput distribution in discriminating and non-discriminating ISPs is similar. Server location is confounding. Simple Discrimination Experiment Long Flow Discrimination Experiment