Presentation is loading. Please wait.

Presentation is loading. Please wait.

Delay Tolerant Bulk Data Transfers on the Internet © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Nikolaos Laoutaris (Telefonica Research)

Published byDeirdre Cameron Modified over 9 years ago

Similar presentations

Presentation on theme: "Delay Tolerant Bulk Data Transfers on the Internet © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Nikolaos Laoutaris (Telefonica Research)"— Presentation transcript:

1 Delay Tolerant Bulk Data Transfers on the Internet © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Nikolaos Laoutaris (Telefonica Research) based on joint work with: Georgios Smaragdakis (DT-Labs), Pablo Rodriguez (Telefonica), Ravi Sundaram (Northeastern)

2 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal This talk is about rethinking of the Internet 2 from a communication network for delay intolerant data into a cargo network for delay tolerant bulk data

3 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Delay Tolerant Bulk (DTB) data Definition: — Delay tolerance ~ few hours to few days — Volume ~ terabytes to petabytes Several examples: — Scientific datasets — Data center synchronization, accounting, data-mining traffic — Distributed production of movies — High resolution images 3

4 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Current situation DTB data are served by: Expensive dedicated networks — LHC Computing Grid Postal system in hard disks & DVDs 4 Our objective is: Utilize leftover capacity from commercial ISPs to send DTB data Can Telco’s compete with ?

5 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal (B) with FedEx International priority — 54 Tbytes every 2 days — ~70 Kgr using 1 Tbyte disks (1.2 Kgr) — $1.2K / shipment  $18K / month Case study: Sending CERN LHC data 5 (A) with a dedicated line — 27 Tbytes/day = ~ 2.5 Gbps — $30K/Gbps/month (Europe/N. America) — $90K/Gbps/month (Asia/S. America) — $200K-$500K / month LHC@CERN Fermilab Push 27 Tbytes/day

6 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Our approach: (C) Use commercial ISPs 6

7 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal How would it work? 7 initially here need to get there DTB databackgound $ $ How can we mininize the cost for ISP(v) and ISP(u)? 95-percentile pricing X(t) volume sent at t (5-minute slots) X vector over a month q 95 (X) c() concave pricing function c(q 95 (X)) monthly charge

8 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Simplest mixing: E2E-CBR B=27 Tbytes, T=1 day  CBR rate B/T=2.5 Gbps traffic ISP(v)  TR day of week background DTB rate B/T background + DTB 95-percentile E2E-CBR as expensive as a dedicated line!

9 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal E2E-Sched traffic ISP(v)  TR valley day of week background 95-percentile Take advantage of Delay Tolerance Regulate source rate and send DTB only during load valleys (already paid for bandwidth)

10 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal “Water-filling” for sending F(q 95 (x),x,t 0,T) of DTB for free! — If x(t) < q 95 (x) add q 95 (x) -x(t)-ε DTB data — If x(t) ≥ q 95 (x) add C-q 95 (x)-Δ DTB data t T x(t) q 95 (x) Similarly, for q>q 95 (x) we can send F(q,x,t 0,T) at cost c(q)-c(q 95 (x)) q 0 The details of source scheduling FREE!

11 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Is this the whole story? No (please don’t try it at home ) Remember there are at least 2 charged links Their valleys are generally non-coinciding

12 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Non-coinciding valleys at sender-receiver Load time-series from 280 interconnection points of a large transit provider Sender in LAT Receiver in EU or China Aggregate results from TR uvalley(LAT) 12 8am 1pm 8pm

13 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal What about Store n’ Forward? 13 E2E-Sched SnF E2E-CBR To recap...

14 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal E2E-Sched vs. SnF(1/5) 14 METHODOLOGY 200 links from a large ISP with global presence For every pair ISP(v), ISP(u) in our dataset We calculate F(E2E-Sched) and F(SnF) — Amount of data that can be sent for free between ISP(v) and ISP(u) in 24 hours — Water-filing: independent (SnF), constrained by min (E2E-Sched)  We report median values for a working week

15 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal E2E-Sched vs. SnF(2/5) 15 T=1 day E2E-Sched SnF SnF (Tbytes) E2E-Sched (Tbytes)

16 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal E2E-Sched vs. SnF(3/5) 16

17 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal E2E-Sched vs. SnF(4/5) 17

18 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal E2E-Sched vs. SnF(5/5) 18

19 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal The cost of meeting deadlines What if we want to send more than what fits for free? To meet the deadline — we need to send faster than what water- filling based on q 95 (x v ), q 95 (x u ) allows — will have to pay extra cost Min cost for sending volume B in [t 0,t 0 +T) under policy P: — Find q v > q 95 (x v ) and q u >q 95 (x u ) — To minimize c(q v )+c(q u )-c(q 95 (x v )) -c(q 95 (x u )) — Subject to F(P,q v,q u,t 0,T)=B q 95 (x v ) q 95 (x u )

20 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Things that affect the minimum cost 20 load ISP(v) → TR load TR → ISP(u) C(q) q bandwidth price ISP(v) → TR C(q) q bandwidth price TR → ISP(u) Data: B Delay Tolerance: T

21 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal E2E-Sched vs. SnF vs. FedEx for CERN 21

22 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Ongoing work 22

23 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Summary 23 E2E-CBR not cheaper than a dedicated link if B<F(E2E-Sched)  send with E2E-Sched at 0 cost If F(E2E-Sched)<B<F(SnF)  send with SnF at 0 cost If B>F(SnF)  send with SnF at minimum transit cost FedEx is cheaper for single shipments but not for flows

24 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal A more general take home message Internet = communications  max delay ~ msec – min  scheduling = congestion avoidance Internet = bulk data dissemination  max delay ~ hours – days  scheduling = network efficiency 24

25 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal More info at: http://research.tid.es/nikos/

26 TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Cost for E2E-Sched for matching F(SnF) 26

Download ppt "Delay Tolerant Bulk Data Transfers on the Internet © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Nikolaos Laoutaris (Telefonica Research)"

Similar presentations

TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Good Things Come to Those Who (Can) Wait or how to handle delay tolerant.

TELEFÓNICA I+D © 2007 Telefónica Investigación y Desarrollo, S.A. Unipersonal Good Things Come to Those Who (Can) Wait or how to handle delay tolerant.
Big Data + SDN SDN Abstractions. The Story Thus Far Different types of traffic in clusters Background Traffic – Bulk transfers – Control messages Active.

Big Data + SDN SDN Abstractions. The Story Thus Far Different types of traffic in clusters Background Traffic – Bulk transfers – Control messages Active.
1 EP2210 Fairness Lecture material: –Bertsekas, Gallager, Data networks, 6.5 –L. Massoulie, J. Roberts, Bandwidth sharing: objectives and algorithms,“

1 EP2210 Fairness Lecture material: –Bertsekas, Gallager, Data networks, 6.5 –L. Massoulie, J. Roberts, "Bandwidth sharing: objectives and algorithms,“
Line Balancing Problem A B C 4.1mins D 1.7mins E 2.7 mins F 3.3 mins G 2.6 mins 2.2 mins 3.4 mins.

Line Balancing Problem A B C 4.1mins D 1.7mins E 2.7 mins F 3.3 mins G 2.6 mins 2.2 mins 3.4 mins.
UDgateway WAN Optimization. 1. Why UDgateway? All-in-one solution Value added services – Networking project requirements Optimize IP traffic on constrained.

UDgateway WAN Optimization. 1. Why UDgateway? All-in-one solution Value added services – Networking project requirements Optimize IP traffic on constrained.
An Analysis of Bulk Data Movement Patterns in Large-scale Scientific Collaborations W. Wu, P. DeMar, A. Bobyshev Fermilab CHEP 2010, TAIPEI TAIWAN

An Analysis of Bulk Data Movement Patterns in Large-scale Scientific Collaborations W. Wu, P. DeMar, A. Bobyshev Fermilab CHEP 2010, TAIPEI TAIWAN
1. Class is already selected! Just press ENTER 2 If you type your I.D. number, press the ENTER key to continue If your name doesn’t show press Cancel.

1. Class is already selected! Just press ENTER 2 If you type your I.D. number, press the ENTER key to continue If your name doesn’t show press Cancel.
James 1:5 If any of you lacks wisdom, he should ask God, who gives generously to all without finding fault, and it will be given to him.

James 1:5 If any of you lacks wisdom, he should ask God, who gives generously to all without finding fault, and it will be given to him.
Vytautas Valancius 1.  When Life Was Simple: Phone Networks  Network of Networks: The Internet  Connectivity structure  Pricing in the Internet: One-Size-Fits-All.

Vytautas Valancius 1.  When Life Was Simple: Phone Networks  Network of Networks: The Internet  Connectivity structure  Pricing in the Internet: One-Size-Fits-All.
Wide Area Networks School of Business Eastern Illinois University © Abdou Illia, Spring 2007 (Week 11, Thursday 3/22/2007)

Wide Area Networks School of Business Eastern Illinois University © Abdou Illia, Spring 2007 (Week 11, Thursday 3/22/2007)
Lecture 2 Introduction 1-1 Chapter 1: roadmap 1.1 What is the Internet? 1.2 Network edge  end systems, access networks, links 1.3 Network core  circuit.

Lecture 2 Introduction 1-1 Chapter 1: roadmap 1.1 What is the Internet? 1.2 Network edge  end systems, access networks, links 1.3 Network core  circuit.
CS335 Networking & Network Administration Tuesday, May 18, 2010.

CS335 Networking & Network Administration Tuesday, May 18, 2010.
Architecture and Routing for NoC-based FPGA Israel Cidon* *joint work with Roman Gindin and Idit Keidar.

Architecture and Routing for NoC-based FPGA Israel Cidon* *joint work with Roman Gindin and Idit Keidar.
Rethinking Internet Traffic Management: From Multiple Decompositions to a Practical Protocol Jiayue He Princeton University Joint work with Martin Suchara,

Rethinking Internet Traffic Management: From Multiple Decompositions to a Practical Protocol Jiayue He Princeton University Joint work with Martin Suchara,
1 Data Storage MICE DAQ Workshop 10 th February 2006 Malcolm Ellis & Paul Kyberd.

1 Data Storage MICE DAQ Workshop 10 th February 2006 Malcolm Ellis & Paul Kyberd.
Using Prices to Allocate Resources at Access Points Jimmy Shih, Randy Katz, Anthony Joseph One Administrative Domain Access Point A Access Point B Network.

Using Prices to Allocate Resources at Access Points Jimmy Shih, Randy Katz, Anthony Joseph One Administrative Domain Access Point A Access Point B Network.
The Delta Routing Project Low-loss Routing for Hybrid Private Networks George Porter (UCB) Minwen Ji, Ph.D. (SRC - HP Labs)

The Delta Routing Project Low-loss Routing for Hybrid Private Networks George Porter (UCB) Minwen Ji, Ph.D. (SRC - HP Labs)
Internet Video: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc.

Internet Video: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc.
Error Checking continued. Network Layers in Action Each layer in the OSI Model will add header information that pertains to that specific protocol. On.

Error Checking continued. Network Layers in Action Each layer in the OSI Model will add header information that pertains to that specific protocol. On.
Global Gophers LLC How the Global Gopher service works…

Global Gophers LLC How the Global Gopher service works…

Similar presentations

Ads by Google