Presentation is loading. Please wait.

Presentation is loading. Please wait.

COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 1 A case study: IPTV SLA Monitoring Giorgio.

Similar presentations


Presentation on theme: "COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 1 A case study: IPTV SLA Monitoring Giorgio."— Presentation transcript:

1 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 1 A case study: IPTV SLA Monitoring Giorgio Ventre The COMICS Research Group @ The University of Napoli Federico II, Corso di Reti di Calcolatori II

2 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 2 Outline  The general problem: SLA, who cares?  A business case for QoS  Defining Service Level Agreements  A Real-Life SLA monitoring service  A case study: IPTV SLA Monitoring

3 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 3 Recent trends in the industry  New emerging multimedia services both in fixed and wireless networks  Traditional voice carriers are moving to NGN: Essential to control costs and drive up revenues Triple play services: Voice – Video – Data Video represents a key element of the service portfolio Price/quality balance must attract/retain users TV quality must compete with satellite and cable

4 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 4 Challenges and quality issues  Users are conditioned to expect high quality TV pictures: Users unlikely tolerate poor/fair quality pictures in IPTV Early delivery of broadband services is unfeasible due to the limited bandwidth compared to cable and satellite Compulsory data compression can potentially degrade quality  Need for robust transmission to minimize data- loss and delay

5 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 5 Why Quality Assurance is a major issue?  Because otherwise we wouldn’t be here  Quality Assurance adds a new perspective to the flatness of the current market of triple-play services  Quality measurement for service assurance End-to-end quality monitoring SLA based on quality delivered to end-user New business models and scenarios

6 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 6 QoS vs QoE  Quality of Service (QoS) refers to the capability of a network to provide better service to selected network traffic over various technologies. QoS is a measure of performance at the packet level from the network perspective.  Quality of Experience (QoE) describes the performance of a device, system, service, or application (or any combination thereof) from the user’s point of view. QoE is a measure of end-to- end performance at the service level from the user perspective.

7 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 7 From QoS to MOS  MOS: Mean Opinion Score  Used in POTS to have a quantitative value for a “qualitative” evaluation: How do you evaluate the quality you perceived during your last service usage/access?  Very easy for simple services: telephony  Very complex for complex services: multimedia (sound vs video vs data vs mix)  Even more complex when quality of service depends on the distribution network AND terminals AND servers

8 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 8 QoS evaluation

9 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 9 Requirements  Identify parameters contributing to a satisfactory QoE  Define network performance requirements to achieve target QoE  Design measurement methods to verify QoE

10 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 10 Performance parameters  IPTV service is highly sensitive to packet loss  The impact of packet loss depends on several factors: Compression algorithm (MPEG2, H.264) GOP structure Type of information lost (I, P, B frame) Codec performance (coding, decoding) Complexity of the video content Error concealment at STB

11 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 11 Quality Measurement  Quality Measurement Objective Pure computational Network performance Objective perceptual Measurements representative of human perception  Traditional metrics such as PSNR, PLR, BER are inadequate  Requirements for objective perceptual metrics

12 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 12 Why Quality-Monitoring is hard?  Measures have to be: Time-based Remoted Distributed Sharp  Highly etherogeneous environments (codecs, CPEs, media-types, …)  Sampled measures? SLAs are not sampled.  In order to ensure quality, measures have to be carried out with quality

13 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 13 Why Quality-Monitoring is hard?  High impact also of content based factors:  MPEG performance depends on content “pattern” and scene changes  Highly variable (movements, colours, lights) scenes generates more data Stallone vs Bergman or better Rambo vs The Seventh Seal

14 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 14 Methods: state of the art  Full-Reference  Reduced-Reference  No-Reference

15 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 15 Full-reference  Measures are performed at both the input to the encoder and the output of the decoder  Both the source and the processed video sequences are available  Requires a reliable communication channel in order to collect measurement data

16 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 16 Reduced-Reference  Extracts only a (meaningful) sub-set of features from both the source video and the received video  A perceptual objective assessment of the video quality is made  The transmitter needs to send extracted features in addition to video data

17 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 17 No-Reference  Perceptual video quality evaluation is made based solely on the processed video sequence  There is no need for the source sequence  Measurements results are intrinsically based on a predictive model

18 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 18 Standards for voice quality assessment  ITU-T P.862 (Feb. 2001): Full-reference perceptual model (PESQ) Signal-based measurement Narrow-band telephony and speech codecs P.862.1 provides output mapping for prediction on MOS scale  ITU-T P.563 (May 2004): No-reference perceptual model Signal-based measurement Narrow-band telephony applications

19 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 19 Standards for voice quality assessment  ITU-T P.862.2 (Nov. 2005): Extension of ITU-T P.862 Wide-band telephony and speech codecs (5 ~ 7Khz)  ITU-T P.VQT (ongoing) Targeted at VoIP applications Uses P.862 as a reference measurement Models analyze packet statistics; speech payload is assumed

20 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 20 Standards for video quality assessment  ITU-T J.144 and ITU-R BT.1683 (2004) Full reference perceptual model Digital TV Rec. 601 image resolution (PAL/NTSC) Bit rates: 768 kbps ~ 5 Mbps Compression errors

21 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 21 Standards for video quality assessment  IETF RFC 4445 (April 2006): A proposed Media Delivery Index (MDI)  MDI can be used as a quality indicator for monitoring a network intended to deliver applications such as streaming media, MPEG video, Voice over IP, or other information sensitive to arrival time and packet loss.  It provides an indication of traffic jitter, a measure of deviation from nominal flow rates, and a data loss at-a-glance measure for a particular video flow.

22 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 22 Our research  Objectives: Real-time computation of achieved quality level “Quality” as perceived by the user Per-single-user measurements Light computation (about +5% overhead)  Approach: Media playout and measures are both part of an integrated process Measurement subsystems exposes a consistent abstract interface Measurements results are high-level quality indicators

23 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 23 VQM (1/2)  No-Reference  Evaluates the video quality as perceived by the user QoS  QoE  Based on MPEG2  Light parsing Doesn’t parse motion vectors, DCT coefficients, and other macroblock-specific information degradation due to packet losses is estimated using only the high-level information contained in Group of Pictures, frame, and slice headers

24 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 24 VQM (2/2)  Does not need to make assumptions concerning how the decoder deals with corrupted information i.e. what kind of error concealment strategy it uses.  Based on this information it determines exactly which slices are lost GoP loss-rate Frame loss-rate Slice loss-rate Differentiation per frame type (I, P, B)  It computes how the error from missing slices propagates spatially and temporally into other slices  Appropriate for measuring video quality in a real-time fashion within a network

25 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 25 Parsing method (1/2) IBBPBBPBBPBB GOP Frame XX

26 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 26 Parsing method (2/2) Quality Measurement 001100101101011010111010101000010101 MPEG-2 video bitstream DECODER Decoded video stream RENDERING HEADERS

27 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 27 QoE vs. MOS  Mapping between Quality of Experience evaluation and MOS (Mean Opinion Score – ITU/T P.800) value QoE MOS Q MAX 5 4 3 2 1

28 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 28 MOS vs SLAs  Knowledge of the function MOS(t) directly enables SLAs monitoring 5 4 3 2 1 MOS TIME DOWN TIME SLA TRESHOLD

29 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 29 Video Server Controlled-Loss Router Video Client + Quality Meter Experimental testbed Video Characteristics: MPEG2-TS Constant Bit Rate: 3.9Mbps Dropped Packets

30 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 30 High Quality Throughput: 5.0 Mbps

31 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 31 Medium Quality Throughput: 3.9 Mbps

32 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 32 Low Quality Throughput: 3.0 Mbps

33 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 33 From SLA to PLA: Provisioning Level Agreements Giorgio Ventre The COMICS Research Group @ The University of Napoli Federico II, & ITEM Laboratory, Italian University Consortium on Informatics Scuola di Dottorato in Ingegneria Informatica Palermo, 17-28 settembre 2007

34 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 34 A service model for resilient networks  We are moving from Quality of Service to a more complex concept of quality+resiliency

35 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 35 Quality of future distributed services  The most important QoS characteristic for future distributed services is arguably going to be resilience  Resilience is the property of a system to restore services to normal after a failure (as fast as the service users need)  However, an investigation into resilience reveals the importance of considering risk when developing our future research agenda

36 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 36 The need for resilience  We are increasingly reliant on the Internet and on networked systems in general (including of course the Web)  This is happening in businesses and indeed in every walk of life including the home  The EU is promoting and developing the Information Society, which is based on communication technologies and systems

37 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 37 Interdependence of networks (1)  Not only are we dependent on networks  But all sorts of other networks are, too Electricity, water, gas Corporate networks Banking networks Health networks …  Information networks are crucial to the successful operation of other networks

38 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 38 Interdependence of networks (2)  Interdependencies of critical infrastructures Power nets and information nets: The virtual utility Rune.Gustavsson@bth.se “The introduction of proper supporting ICT of power nets forming a virtual utility is an important instance of networked enabled capabilities (NEC) systems. Furthermore, by pursuing this task we can gain experiences and develop models and technologies that besides addressing societal critical systems also can be useful in other efforts on development and maintenance of complex systems.”  In Italy, Report del Comitato sulla Protezione delle Infrastrutture Critiche, Presidenza del Consiglio dei Ministri, 2004

39 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 39 Internet meltdown?  Article in The Independent (UK) 8 September 2004: “The internet is becoming a utility” [Karl Auerbach] As a utility, the net will have to live up to different, more stringent standards than its previous uses as an academic and research playground, and then a mainstream experiment. People are building billion- dollar businesses, governments are turning themselves digital, and in the meantime there isn't so much as a service-level agreement to guarantee that the most basic level of connectivity will be there tomorrow. If the technologists no longer believe they can fix it by themselves, the Internet really has hit a meltdown.

40 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 40 Vulnerabilities  The Internet was originally designed to withstand basic link and switch failures  But it was never envisaged as a utility (i.e. offering near- perfect availability), supporting commercial initiatives and acting as a vital infrastructure  Whatever vulnerabilities are present in the infrastructure may be inherited by the applications it aims to support

41 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 41 Attacks  Complex, well engineered systems should be built by keeping in mind faults  Today, we need to keep into account other disruption sources  Network attacks of all sorts are increasing in variety and number: Spam / junk Viruses, Worms etc. DDoS attacks Physical …  These cause huge costs in time and energy, but no coherent approach to a solution

42 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 42 Multiple levels  This is of course a multi-level problem Physical layer Networking / IP Middleware layer / O.S. Web / applications  A solution to achieving resilience needs to apply at all levels: this is a grand challenge for future networked systems infrastructure

43 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 43 Complexity  This is a distributed computing problem  According to Leonard Kleinrock, we have no suitable theory to handle this, because of its inherent complexity  This is compounded by nomadicity [complicated = difficult to study but fit for purpose, static; whereas complex = growing, evolving]

44 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 44 Complexity not simplicity  In spite of all hype on global network architectures, today we face a complex, heterogeneous reality: Fixed access networks: POTS, xDSL, CATV, MetroLAN Mobile, wireless access networks: GPRS, UMTS, WiFi, Wimax Interoperability with terrestrial digital broadcasting  Additional complexity issues: New, diverse terminals: (Symbian Cell. Phones, PDAs, smart set-top-boxes) Dynamic creation of novel services and applications

45 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 45 Complexity as an opportunity  The availability of a multiplicity of networks, devices and services should be seen as an opportunity: No single infrastructure of critical importance Ease of access to all players: government, companies, common people Availability of a multitude of sources of information Availability of a multitude of computing resources Availability of a multitude of communication media/networks  … provided that such a rich scenario can be managed as a system

46 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 46 Some recent events  We learned some lessons recently: 9/11 2001 Attacks US East Coast Blackout Italy Blackout Series of attacks: Worms (NIMDA, Witty, Slammer …) DDOS Routing attacks  We probably need to re-discover traditional values typical of traditional engineering practice

47 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 47 Lessons from 9/11 2001  “The Internet under Crisis Conditions” A Committee of the National Research Council of the National Academies (www.nap.edu)  Findings of the Committee Attacks had very limited effects on the Internet as a global, best effort communication system Internet technology appears to be robust per se but considerable efforts are needed to protect Internet- based systems Many critical interdependencies discovered only after the attacks

48 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 48 Known and less known effects  Dependency of Internet on other telecommunication systems (fixed, wireless, cellular) Obvious: co-location of sites, tubes, cables; running out of diesel… Not so obvious : e.g. communications between NYC ISPs and TelCos hampered by problems to toll-free numbers  Facility disaster planning as a rare expertise/culture in the Internet world Very limited capacity of backup power generation even in major ISP sites/POPs  Other issues, e.g. DNS for.za domain was hosted on a server in NYC WiFi LANs of two major Manhattan hospitals operating in outsourcing via Internet

49 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 49 Lessons from 9/28 2003  Anticipated by the US East Coast blackout: much larger scale than WTC but apparently more limited damage  Different effects and impacts POTS infrastructure capable of enduring very long power outages: practically no effects Cellular Networks locally in deep crisis National TV and Radio broadcasters OK, local players generally in crisis “Global” and VoIP operators knocked-out  What about the Internet? All IT based services affected : AAA, CDN, Servers

50 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 50 Lessons in ATC systems  Press Releases (http://www.natca.org/mediacenter/press-release- detail.aspx?id=394)  MASSIVE POWER, COMMUNICATIONS FAILURE AT MAJOR AIR TRAFFIC CONTROL CENTER PUTS CONTROLLERS IN DARK, FLIGHTS IN JEOPARDY  07/19/2006 Bob Marks PALMDALE, Calif. – A massive power and communications failure late Tuesday at the Los Angeles Air Route Traffic Control Center left scrambling air traffic controllers to deal with a nightmare scenario – how to keep dozens of flights away from each other above a large swath of the Southwestern United States despite the inability to see them, talk to them or relay crucial instructions for 15 excruciatingly long minutes. Every ounce of skill, heart and determination that controllers bring into the control room every day was put to the test during one of the worst outages to ever hit the facility. It was so bad, controllers say, that the only thing they had of use to aid the situation that actually worked was their cell phones – devices which the Federal Aviation Administration, inexplicably, has barred from control rooms, further impeding the safety of the system.  More details in http://themainbang.typepad.com/blog/2006/07/complete_failur.html

51 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 51 Issues for research (1)  Forget OSI-type layering/abstractions Services depend not only on peer and adjacent layers Resiliency is a system-wide issue, with vertical and horizontal dependencies Start speaking about networked systems and not only of networks IT based services must be considered as part of the whole picture Contributions from several disciplines Multi-level approach Cross-layer approach

52 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 52 Issues for research (2)  Monitoring of services and infrastructures We can’t trust what we can’t control  Robustness of services To unexpected situations: faults, misconfigurations, excessive demand, soft attacks (DDOS) To expected but complex situations: tools/methodologies for proper dimensioning of services (Service Engineering)  Resiliency of infrastructures Focus on survivability of communication systems to hard attacks (terrorist hits, natural disasters)  Reconfigurability of communication systems Make different networks/systems a single infrastructure

53 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 53 Issues for research (3)  Towards a GRID of communication infostructures Connect them all physically Make them resilient separately Allow for services to migrate Prepare for interconnecting them if needed  From the computational GRID to the communication GRID But try to make it with an autonomic communication flavour

54 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 54 Issues for research  Resiliency of infrastructures Focus on survivability of communication systems to hard attacks (terrorist hits, natural disasters)  Reconfigurability of communication systems Make different networks a single infrastructure  Resiliency of services To unexpected situations: faults, excessive demand, soft attacks (D-DOS) To expected but complex situations: tools/methodologies for proper dimensioning of services (Service Engineering)

55 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 55 From QoS to Resiliency to …  We should not forget the past  QoS is as important as resiliency, and is back “Value of supporting Class-of-Services in IP Backbones”, M. Yuksel et al., IWQOS 2007  Also because QoS is a good mechanism to improve resiliency of a distributed system  So, we should probably talk about QoSiliency

56 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 56 User-Centered Architectures Service Controllers Access Controllers Service Directories Resource Controllers QoS-capable Networks User Commercial Service

57 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 57 Service Controller Access Controllers User Service Directories QoS-capable Networks SLA SLS Resource Controllers Info about content (metadata) z --- Policy rules SLAs are the triggers

58 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 58 A change of perspective  One of the major problems with SLA based architectures was their limited capability to scale with the number of users and services  We therefore introduce the concept of Provisioning Level Agreement (PLA): A PLA is a contract between a service provider and the owner of the Infrastructure defining the level of service to be guaranteed to final users during the provisioning of a service on top of that Infrastructure.

59 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 59 A change of perspective (cont.)  In a PLA it is the service provider who defines the type of service the treatment the service needs to get from the network (QoS, resiliency needs, security and privacy reqs.) the classes of possible SLAs that can be subscribed by the users  A PLA is signed at service deployment time, and can be dynamically modified and updated any time the service characteristics and requirements change  Once a PLA is signed, Provisioning Level Specifications are produced to allow the infrastructure to be properly configured to accommodate the new service and future service subscriptions by final users

60 COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 60 Service Provider Service Directories QoS-figurable Networks PLA PLS Resource Controllers Info about content (metadata) z --- Policy rules PLAs are the triggers Service Centered Architectures


Download ppt "COMICS (COMputer for Interaction and CommunicationS) Research Group – DIS, University of Napoli Federico II 1 A case study: IPTV SLA Monitoring Giorgio."

Similar presentations


Ads by Google