Presentation is loading. Please wait.

Presentation is loading. Please wait.

Continuous analytics for traffic monitoring and applications to CDN The need of an intelligent measurement plane for the internet CoNEXT TPC workshop Barcellona,

Similar presentations


Presentation on theme: "Continuous analytics for traffic monitoring and applications to CDN The need of an intelligent measurement plane for the internet CoNEXT TPC workshop Barcellona,"— Presentation transcript:

1 Continuous analytics for traffic monitoring and applications to CDN The need of an intelligent measurement plane for the internet CoNEXT TPC workshop Barcellona, 5/9/13

2 2/26 Something we watch 5-May6-May7-May8-May9-May10-May SUBNETNAME with AIRPORT code#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg 173.194.18fra02s08.c.youtube.com 6139368.936266298.45389228.76 173.194.19fra02s15.c.youtube.com 9940258.1812893196.067012166.48 173.194.2mil01s12.c.youtube.com170541333.46154701276.31136551259.63141861296.07126161197.64138601338.63 173.194.20par08s06.c.youtube.com 173.194.208par08s06.c.youtube.com 487414.6371711.54 173.194.5lhr14s08.c.youtube.com4491819.572831658.45 3470937.1842221025.4961911166.08 173.194.6fra07s13.c.youtube.com 4924412.178749331.147224318.83 173.194.62fra07s19.c.youtube.com 6160325.826877248.396108274.55 173.194.9par03s06.c.youtube.com 87355.1 208.117.236par03x04.c.youtube.com179164.184250540.16957496.91 208.117.248mia02s11.c.youtube.com 77552 208.117.250ams09x06.c.youtube.com4143067949437656.3957675653.81567906.65 208.117.252dfw06x02.c.youtube.com 51285.63 208.117.254fra07x03.c.youtube.com838667.292130852.53 465606.11261146.8710331379.76 74.125.105lhr22s16.c.youtube.com18291551.7816551185.943957942.473454990.6441161061.7276571126.83 74.125.13zrh04s03.c.youtube.com7191074.154992264.09821302.03 74.125.14mil02s01.c.youtube.com483661234.82379681253.01371821162.85478441298.45525941226.85377551143.37 74.125.216bru02t11.c.youtube.com 74.125.218fra07t13.c.youtube.com86971355.33125791338.7185601239114691256.32116331292.58103201276.33 74.125.4lhr22s11.c.youtube.com14961846.2524881034.7841461363.63 74.125.99fra07s03.c.youtube.com 4221187.844913189.642461136.63 ISPs

3 3/26 Something we observe 5-May6-May7-May8-May9-May10-May SUBNETNAME with AIRPORT code#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg 173.194.18fra02s08.c.youtube.com 6139368.936266298.45389228.76 173.194.19fra02s15.c.youtube.com 9940258.1812893196.067012166.48 173.194.2mil01s12.c.youtube.com170541333.46154701276.31136551259.63141861296.07126161197.64138601338.63 173.194.20par08s06.c.youtube.com 173.194.208par08s06.c.youtube.com 487414.6371711.54 173.194.5lhr14s08.c.youtube.com4491819.572831658.45 3470937.1842221025.4961911166.08 173.194.6fra07s13.c.youtube.com 4924412.178749331.147224318.83 173.194.62fra07s19.c.youtube.com 6160325.826877248.396108274.55 173.194.9par03s06.c.youtube.com 87355.1 208.117.236par03x04.c.youtube.com179164.184250540.16957496.91 208.117.248mia02s11.c.youtube.com 77552 208.117.250ams09x06.c.youtube.com4143067949437656.3957675653.81567906.65 208.117.252dfw06x02.c.youtube.com 51285.63 208.117.254fra07x03.c.youtube.com838667.292130852.53 465606.11261146.8710331379.76 74.125.105lhr22s16.c.youtube.com18291551.7816551185.943957942.473454990.6441161061.7276571126.83 74.125.13zrh04s03.c.youtube.com7191074.154992264.09821302.03 74.125.14mil02s01.c.youtube.com483661234.82379681253.01371821162.85478441298.45525941226.85377551143.37 74.125.216bru02t11.c.youtube.com 74.125.218fra07t13.c.youtube.com86971355.33125791338.7185601239114691256.32116331292.58103201276.33 74.125.4lhr22s11.c.youtube.com14961846.2524881034.7841461363.63 74.125.99fra07s03.c.youtube.com 4221187.844913189.642461136.63

4 4/26 Something we observe 5-May6-May7-May8-May9-May10-May SUBNETNAME with AIRPORT code#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg 173.194.18fra02s08.c.youtube.com 6139368.936266298.45389228.76 173.194.19fra02s15.c.youtube.com 9940258.1812893196.067012166.48 173.194.2mil01s12.c.youtube.com170541333.46154701276.31136551259.63141861296.07126161197.64138601338.63 173.194.20par08s06.c.youtube.com 173.194.208par08s06.c.youtube.com 487414.6371711.54 173.194.5lhr14s08.c.youtube.com4491819.572831658.45 3470937.1842221025.4961911166.08 173.194.6fra07s13.c.youtube.com 4924412.178749331.147224318.83 173.194.62fra07s19.c.youtube.com 6160325.826877248.396108274.55 173.194.9par03s06.c.youtube.com 87355.1 208.117.236par03x04.c.youtube.com179164.184250540.16957496.91 208.117.248mia02s11.c.youtube.com 77552 208.117.250ams09x06.c.youtube.com4143067949437656.3957675653.81567906.65 208.117.252dfw06x02.c.youtube.com 51285.63 208.117.254fra07x03.c.youtube.com838667.292130852.53 465606.11261146.8710331379.76 74.125.105lhr22s16.c.youtube.com18291551.7816551185.943957942.473454990.6441161061.7276571126.83 74.125.13zrh04s03.c.youtube.com7191074.154992264.09821302.03 74.125.14mil02s01.c.youtube.com483661234.82379681253.01371821162.85478441298.45525941226.85377551143.37 74.125.216bru02t11.c.youtube.com 74.125.218fra07t13.c.youtube.com86971355.33125791338.7185601239114691256.32116331292.58103201276.33 74.125.4lhr22s11.c.youtube.com14961846.2524881034.7841461363.63 74.125.99fra07s03.c.youtube.com 4221187.844913189.642461136.63

5 5/26 Something we observe 5-May6-May7-May8-May9-May10-May SUBNETNAME with AIRPORT code#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg 173.194.18fra02s08.c.youtube.com 6139368.936266298.45389228.76 173.194.19fra02s15.c.youtube.com 9940258.1812893196.067012166.48 173.194.2mil01s12.c.youtube.com170541333.46154701276.31136551259.63141861296.07126161197.64138601338.63 173.194.20par08s06.c.youtube.com 173.194.208par08s06.c.youtube.com 487414.6371711.54 173.194.5lhr14s08.c.youtube.com4491819.572831658.45 3470937.1842221025.4961911166.08 173.194.6fra07s13.c.youtube.com 4924412.178749331.147224318.83 173.194.62fra07s19.c.youtube.com 6160325.826877248.396108274.55 173.194.9par03s06.c.youtube.com 87355.1 208.117.236par03x04.c.youtube.com179164.184250540.16957496.91 208.117.248mia02s11.c.youtube.com 77552 208.117.250ams09x06.c.youtube.com4143067949437656.3957675653.81567906.65 208.117.252dfw06x02.c.youtube.com 51285.63 208.117.254fra07x03.c.youtube.com838667.292130852.53 465606.11261146.8710331379.76 74.125.105lhr22s16.c.youtube.com18291551.7816551185.943957942.473454990.6441161061.7276571126.83 74.125.13zrh04s03.c.youtube.com7191074.154992264.09821302.03 74.125.14mil02s01.c.youtube.com483661234.82379681253.01371821162.85478441298.45525941226.85377551143.37 74.125.216bru02t11.c.youtube.com 74.125.218fra07t13.c.youtube.com86971355.33125791338.7185601239114691256.32116331292.58103201276.33 74.125.4lhr22s11.c.youtube.com14961846.2524881034.7841461363.63 74.125.99fra07s03.c.youtube.com 4221187.844913189.642461136.63

6 6/26 Something we observe 5-May6-May7-May8-May9-May10-May SUBNETNAME with AIRPORT code#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg 173.194.18fra02s08.c.youtube.com 6139368.936266298.45389228.76 173.194.19fra02s15.c.youtube.com 9940258.1812893196.067012166.48 173.194.2mil01s12.c.youtube.com170541333.46154701276.31136551259.63141861296.07126161197.64138601338.63 173.194.20par08s06.c.youtube.com 173.194.208par08s06.c.youtube.com 487414.6371711.54 173.194.5lhr14s08.c.youtube.com4491819.572831658.45 3470937.1842221025.4961911166.08 173.194.6fra07s13.c.youtube.com 4924412.178749331.147224318.83 173.194.62fra07s19.c.youtube.com 6160325.826877248.396108274.55 173.194.9par03s06.c.youtube.com 87355.1 208.117.236par03x04.c.youtube.com179164.184250540.16957496.91 208.117.248mia02s11.c.youtube.com 77552 208.117.250ams09x06.c.youtube.com4143067949437656.3957675653.81567906.65 208.117.252dfw06x02.c.youtube.com 51285.63 208.117.254fra07x03.c.youtube.com838667.292130852.53 465606.11261146.8710331379.76 74.125.105lhr22s16.c.youtube.com18291551.7816551185.943957942.473454990.6441161061.7276571126.83 74.125.13zrh04s03.c.youtube.com7191074.154992264.09821302.03 74.125.14mil02s01.c.youtube.com483661234.82379681253.01371821162.85478441298.45525941226.85377551143.37 74.125.216bru02t11.c.youtube.com 74.125.218fra07t13.c.youtube.com86971355.33125791338.7185601239114691256.32116331292.58103201276.33 74.125.4lhr22s11.c.youtube.com14961846.2524881034.7841461363.63 74.125.99fra07s03.c.youtube.com 4221187.844913189.642461136.63

7 7/26 Something we observe 5-May6-May7-May8-May9-May10-May SUBNETNAME with AIRPORT code#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg 173.194.18fra02s08.c.youtube.com 6139368.936266298.45389228.76 173.194.19fra02s15.c.youtube.com 9940258.1812893196.067012166.48 173.194.2mil01s12.c.youtube.com170541333.46154701276.31136551259.63141861296.07126161197.64138601338.63 173.194.20par08s06.c.youtube.com 173.194.208par08s06.c.youtube.com 487414.6371711.54 173.194.5lhr14s08.c.youtube.com4491819.572831658.45 3470937.1842221025.4961911166.08 173.194.6fra07s13.c.youtube.com 4924412.178749331.147224318.83 173.194.62fra07s19.c.youtube.com 6160325.826877248.396108274.55 173.194.9par03s06.c.youtube.com 87355.1 208.117.236par03x04.c.youtube.com179164.184250540.16957496.91 208.117.248mia02s11.c.youtube.com 77552 208.117.250ams09x06.c.youtube.com4143067949437656.3957675653.81567906.65 208.117.252dfw06x02.c.youtube.com 51285.63 208.117.254fra07x03.c.youtube.com838667.292130852.53 465606.11261146.8710331379.76 74.125.105lhr22s16.c.youtube.com18291551.7816551185.943957942.473454990.6441161061.7276571126.83 74.125.13zrh04s03.c.youtube.com7191074.154992264.09821302.03 74.125.14mil02s01.c.youtube.com483661234.82379681253.01371821162.85478441298.45525941226.85377551143.37 74.125.216bru02t11.c.youtube.com 74.125.218fra07t13.c.youtube.com86971355.33125791338.7185601239114691256.32116331292.58103201276.33 74.125.4lhr22s11.c.youtube.com14961846.2524881034.7841461363.63 74.125.99fra07s03.c.youtube.com 4221187.844913189.642461136.63

8 8/26 Something we observe 5-May6-May7-May8-May9-May10-May SUBNETNAME with AIRPORT code#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg#flowTru avg 173.194.18fra02s08.c.youtube.com 6139368.936266298.45389228.76 173.194.19fra02s15.c.youtube.com 9940258.1812893196.067012166.48 173.194.2mil01s12.c.youtube.com170541333.46154701276.31136551259.63141861296.07126161197.64138601338.63 173.194.20par08s06.c.youtube.com 173.194.208par08s06.c.youtube.com 487414.6371711.54 173.194.5lhr14s08.c.youtube.com4491819.572831658.45 3470937.1842221025.4961911166.08 173.194.6fra07s13.c.youtube.com 4924412.178749331.147224318.83 173.194.62fra07s19.c.youtube.com 6160325.826877248.396108274.55 173.194.9par03s06.c.youtube.com 87355.1 208.117.236par03x04.c.youtube.com179164.184250540.16957496.91 208.117.248mia02s11.c.youtube.com 77552 208.117.250ams09x06.c.youtube.com4143067949437656.3957675653.81567906.65 208.117.252dfw06x02.c.youtube.com 51285.63 208.117.254fra07x03.c.youtube.com838667.292130852.53 465606.11261146.8710331379.76 74.125.105lhr22s16.c.youtube.com18291551.7816551185.943957942.473454990.6441161061.7276571126.83 74.125.13zrh04s03.c.youtube.com7191074.154992264.09821302.03 74.125.14mil02s01.c.youtube.com483661234.82379681253.01371821162.85478441298.45525941226.85377551143.37 74.125.216bru02t11.c.youtube.com 74.125.218fra07t13.c.youtube.com86971355.33125791338.7185601239114691256.32116331292.58103201276.33 74.125.4lhr22s11.c.youtube.com14961846.2524881034.7841461363.63 74.125.99fra07s03.c.youtube.com 4221187.844913189.642461136.63

9 9/26 A complicated technology… The internet is a key infrastructure where different technologies are combined to offer a plethora of services. Its horribly complicated. We sorely miss the technology to understand what is happening in the network and to optimize its performance and utilization.

10 10/26 mPlane mPlane is an FP7 Integrated Project Large scale Collaborative About the design and demonstration of an intelligent measurement plane for the Internet 3 years 16 partners 11.2+Meuros

11 11/26 Who we are

12 12/26 mPlane architecture CoNEXT TPC workshop Barcellona, 5/9/13

13 13/26 CoNEXT TPC workshop Barcellona, 5/9/13

14 14/26 CoNEXT TPC workshop Barcellona, 5/9/13

15 15/26 CoNEXT TPC workshop Barcellona, 5/9/13

16 16/26 CoNEXT TPC workshop Barcellona, 5/9/13

17 17/26 CoNEXT TPC workshop Barcellona, 5/9/13

18 18/26 CDN makes complicated things We have seen this for … … the same applies for Akamai, even in shorter time frames… CoNEXT TPC workshop Barcellona, 5/9/13

19 19/26 Reasoning about the problem How to investigate this kind of events? You need a flexible, scalable, intelligent supervisor and repository DBStream [1] : Continuous big data analytics Flexible processing language Full SQL processing capabilities Processing in small batches Storage for post-mortem analysis CoNEXT TPC workshop Barcellona, 5/9/13 [1] A.Bar, A. Finamore, I. Bermudez, L. Golab, M.Mellia, P.Casas, Continuous analytics for traffic monitoring and applications to CDN, Under submission

20 20/26 Reasoning about the problem Is the change related to single servers? Is this affecting only some services? Was this triggered by CDN performance issues? … CoNEXT TPC workshop Barcellona, 5/9/13

21 21/26 Reasoning about the problem Is the change related to single servers? Compute the traffic volume per IP address for every 15m time intervals CoNEXT TPC workshop Barcellona, 5/9/13 NO

22 22/26 Reasoning about the problem Is this affecting only some services (*) ? Select the top 500 services served by Akamai Order by frequency Repeat for each 5m time interval NO (*) Service == FQDN

23 23/26 Does this have effects on the QoE? Was this triggered by CDN performance issues? For services served by Akamai preferred cache Compute the distribution of server elaboration time Plot percentiles every 5m of time CoNEXT TPC workshop Barcellona, 5/9/13 YES!! NO!!!

24 24/26 Reasoning about the problem Is the change related to single servers? Is this affecting only some services? Was this triggered by CDN performance issues? … … Any good suggestion?!?!? CoNEXT TPC workshop Barcellona, 5/9/13 NO

25 25/26 Conclusions/WIP We are working to design an intelligent measurement plane We are now working on a scalable repository And on an (un-)supervised supervisor that can (automatically) learn and find the root cause of events We are open to collaborations => ask me CoNEXT TPC workshop Barcellona, 5/9/13

26 Project acronym: mPlane Project full title: mPlane – an Intelligent Measurement Plane for Future Network and Application Management Grant agreement no: 318627 Staring Date: November 1 st 2012 Total Cost: 11,274,908.00Euros Duration: 3 years Partners: 16 Coordinator: Prof. Marco Mellia – Politecnico di Torino - IT


Download ppt "Continuous analytics for traffic monitoring and applications to CDN The need of an intelligent measurement plane for the internet CoNEXT TPC workshop Barcellona,"

Similar presentations


Ads by Google