1. Prenume NUME Funcţie Nume eveniment Oraş, Data NET GRAF Internet QoS measurement tool for end-users Costin MASILIEV Head of Electronic Communications Security Unit, ANCOM Regional Development Forum for Europe (RDF-EUR) Broadband for Sustainable Development Bucharest, 21.04.2015

2. Legal basis ANCOM decision 1201/2011 on establishing the quality indicators for the provision of the Internet access service and the publication of the due parameters D1201/2011 establishes: a relevant set of quality indicators related to the provision of the Internet access service; the conditions and means of publication by the providers of the parameters of these indicators. D1201/2011 specifies that: ANCOM shall create, manage and make available to the public an interactive application on its website to measure the technical quality indicators; The values of the measured quality parameters shall be calculated upon using a test server placed in an interexchange Internet connection node; The interactive application shall be developed in consultation with the providers of internet access services. The aim - to enable the end-users to benefit from relevant, comprehensive, comparable and user-friendly information.

3. Developping and launching the application ANCOM, in consultation with providers, has conducted a public acquisition procedure for implementing a solution for internet access quality measuring; The winner of the tender was designated Industrial Software, a Romanian company delivering integrated software solutions for organizations; Industrial Software has designed, developed, tested and implemented the application. Netograf was launched to the public by ANCOM in october 2014; The application is available to all internet access service users on www.netograf.ro. www.netograf.ro

4. Measurement tool for end-users General features Transparent solution provides transparent information the same measurement conditions for all users test results and statistical results on the quality of services are offered to users in a uniform manner, on comparable bases Easy to use, easy to understand web based application for testing and monitoring the quality of the Internet access service available to the Romanian users the application is based on the principle of crowdsourcing Robust solutionShort measurement timeFlexible solution can run on different devices – fixed and mobile terminals, OS, browser types can be used for mobile and fixed Internet More awareness and knowledge on Internet service quality users can assess the performance achieved by the providers and its evolution over time, i.e. improvement or degradation of service users can compare the results to those in the provider’s commercial offer users can compare the results of one or more providers

5. QoS parameters for measuring internet access performances Quality parameters measured by Netograf are: Download speed (Mbps) Upload speed (Mbps) Latency (ms) Jitter (ms) Packet loss ratio (%)

6. Measuring Procedure Download speed It is measured over multiple threads using a http client to use the whole internet connection. The testing engine automatically detects the number of threads that can be used with the browser; it tries to download around 10 packets per second per thread. Upload speed The engine uses a set of configuration parameters used in the same way as for the download. The packets are created on the client side and then sent to the test servers on multiple threads. The size of the packets is configured as a percentage of the packet used during download. After the test is finished, the engine calculates the average speed by counting the packets transferred successfully in the time given. The platform allows configuring the number of packets that can be excluded from the calculation, both as a percentage of the worst results and the fastest ones. The length of the test (the number of seconds) can be configured.

7. Measuring Procedure Latency The test uses a number of 10 http ping packets (the number can be configured). From the 10 packets sent, the best and worst results are excluded and the rest is averaged to get the result. The latency is calculated in the application layer and estimates the response time from the client to the server side. Jitter For each packet sent (http ping packets), it is calculated the difference in milliseconds between consecutive packets. From the 10 packets, the best and worst results are excluded and the rest is averaged to get the result. Packet loss The engine opens a number of five parallel sessions and sends ICMP ping commands. The number of packets is set to 100 packets (can be configured). The testing engine then measures the number of the packets received.

8. Hardware architecture scheme of the measurement platform The platform is connected to an internet interexchange point (IXP) through a Cisco ASR 1006 Router.

9. Other technical aspects The platform has three main modules: the testing module; the providers module; the administration module. The application uses functions that are available in the application layer; The application uses PHP as programming language; The testing engine is developed in javascript; The testing algorithm is run by the user browser to avoid overloading the server and to perform a precise measurement.

10. Performing a test Some data is automatically filled before the test is started, after pressing the Testing button: IP address of the user; The user's location (county, town and locality) - according to GeoIP database; Browser type and version; Operating system; Terminal type (computer/mobile terminal); The name of the ISP, based on the IP address; if no IP range is found in the application database, GeoIP database is interrogated in order to detect the provider; the user can change the provider’s name if necessary.

11. Performing a test In order to start a test, a user has to pick the internet offer he owns from the list provided by the Internet service provider; if the user doesn’t know which offer he has in his internet contract, she/he can pick the option ”Unknown”; When the test is launched, the user is asked how the internet is accessed; the question is based on the provider type: mobile service provider or fixed line service provider: Fixed line service provider Wireless Wired Mobile service provider Indoor Outdoor

12. Test results After completing the set of information, the test runs and the user can view the test results and can access some additional information on the average values of the parameters tested per provider, type of offer, location of the test.

13. Additional information after performing a test

14. Publication of comparative statistical values Based on the tests performed by the users, the application also calculates: the number of tests performed the average values of the technical quality parameters These statistical values are presented separately for fixed internet and mobile internet; A user can compare the results of one or more providers, being able to add maximum 4 selection panels for this purpose; Statistical values are available: quarterly - results encompass a quarter yearly - results are updated in real time for a certain provider per access type (wired/wireless or indoor/outdoor)

15. Publication of comparative statistical values The application calculates for each provider the average values registered by the provider in different scenarios: irrespective of offer type or location; per offer; per location (county, town, locality); per offer and location. For the accuracy of the measurements included in these cumulative results, tests are subject to some exclusion rules. Having such additional information, the users are able to evaluate the network performance as experienced by other users.

16. Publication of comparative statistical values Comparative statistical values registered by the selected providers irrespective of offer type or location

17. Publication of comparative statistical values Comparative statistical values registered by the selected providers per offer type and location (county, town and locality)

18. History of user’s results The user can log to the portal and has the ability to manage saved internet accounts and create new accounts; Logged user can access the history of his results.

19. Next steps Platform administration Monitoring of the application; Optimization of the application; Adding the commercial offers and IP ranges in the application; Elaboration of reports etc.. Platform extension Developing and testing of new capabilities (in terms of hardware and software).

20. Thank you for your attention! costin.masiliev@ancom.org.ro