1. www.seeren.org SEEREN The SEEREN initiative is co-funded by the European Commission under the FP5 contract IST-2001-38830 The Virtual Network Operations Center and Network Management Framework in SEEREN Octavian Rusu, Valeriu Vraciu - RoEduNet Jorge-A. Sanchez-P., Constantinos Kotsokalis - GRNET Hristo Turlakov, Vedrin Jeliazkov - ISTF Slavko Gajin - AMREJ, Neki Frasheri - INIMA, Goran Muratovski - MARNET

2. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 2 SEEREN South Eastern European segment of the multi-gigabit pan- European Research and Education network, GÉANT. SEEREN interconnects the Research and Education Networks of: Albania – INIMA Bosnia-Herzegovina – BIHARNET Bulgaria – ICTDA Greece – GRNET FYRoM – MARNET Hungary – HUNGARNET Romania – RoEduNet Serbia-Montenegro – AMREJ SEEREN infrastructure was launched and entered its stable operation on January 2004

3. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 3 SEEREN Network Topology Links to MPLS Network: INIMA – 2 Mbps BIHARNET – 2Mbps ICTDA – 18-34 Mbps GRNET – 155 Mbps (main connection to GEANT) MARNET – 2x2Mbps RoEduNet – 34 Mbps (backup connection to GEANT) AMREJ – 34 Mbps

4. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 4 Network Management Network management the mechanism used for all managed objects within the Network Layers of the OSI model to do monitoring, controlling, coordination Strategies: Centralized Distributed Hierarchical Network management components: Configuration management - detects and controls the state of the network; Performance management - controls and analyses throughput and error rate; Fault management - is responsible for detecting, isolating and controlling abnormal behavior; Accounting management - collects and processes data about resource consumption in the network; Security management - deals with access control Service management components: Monitoring - involves gathering data about the network; Control - manipulation of devices; Reporting - abnormal events are reported

5. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 5 SEEREN Virtual NOC Framework NME Network Management Entity HelpDesk SIE Service Implementer Entity SEEREN NOC Network Operation Center APMs Access Port Managers Operator Carrier/Service Providers PSC Project Steering Committee NME Help Desk Operator # SIE # APMs NOCs PSC

6. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 6 NME - Network Management Entity The technical core of the management team for the entire network proposes the main network policies, including the network evolution and upgrades of equipments and services to PSC; performs the high level design of all services; decides about special solutions and services by appropriate Service Implementer Entity (SIE); coordinates the Help Desk activities responsible for the technical integrity of the services provided on the network; implements new services using configuration solutions provided by SIEs; technically defines and modifies network policies; plans network development; operates the Help Desk, which interacts with: APMs; Operators, to provide fault isolation and management of the links and/or services supervised by a different authority; SIEs, during testing period for new services. operates the Trouble Ticket System

7. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 7 SIEs - Service Implementer Entities Specialized task teams distributed in the service dimension provide studies for proposed services by NME, specifying issues of interest for the network objectives and policies; provide configuration files for network equipment to implement the proposed services; interact with NME during service activation; report through the Help Desk problems related to a service; monitor service operation using network management tools during the implementation period.

8. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 8 APMs – Access Port Managers Manager of geographically distributed teams (one for each NOC) responsible for the local NOC activities monitor the network operation in their area of authority; configure the local communications equipment; monitor the implementation of the services within their NOCs; interact with NME to maintain the centralized management system; interact with the users at the NOC level.

9. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 9 Virtual NOC Advantages Centralized character for network operation all information flows through the NME. Distributed and hierarchical character achieved trough: APMs provide network management and user support within a geographical area of authority SIEs responsible for particular services implementation on the entire network. interaction between SIEs and APMs is handled by NME providing consistency of all operations. operation of NOCs and even the service implementation procedures are distributed and can be outsourced Establishes the responsibilities of each unit involved in the management process Precise split of functions to different groups and users Can be implemented using a mix of distributed and centralized strategies Implementation – software tools: centralized and distributed components.

10. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 10 SEEREN Management Tools NetIS - The Network Information System - developed by AMREJ and hosted at Belgrade University Computing Center Nagios - host, service and network monitoring software running on central management server Looking Glass : Tools for fast web-based (read-only) access to the routers Helpdesk and Trouble Ticket System - The SEEREN Helpdesk provides services only to the SEEREN NREN NOCs. Other Tools: RANCID - used for the CVS repository of router configuration SmokePing - network latency monitor which works in a way that is similar to MRTG WeatherMap - perl tool that displays in a web page the utilization of the network links One-stop-shop for all tools at http://admin.seeren.org

11. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 11 NetIS Monitors the status of the network nodes, lines and services; Traffic accounting and reporting; Read-only access to network routers; Traffic monitoring for network threats and attacks response; Web-based user access

12. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 12 NetIS SEEREN from the sky

13. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 13 NetIS SEEREN monitors

14. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 14 Nagios Centralized monitoring Specialized tools for monitoring hosts; routers; resources; network services (HTTP, SMTP, FTP). Features: contact notifications - email; ability to define event handlers (interact with TTS); scheduled downtime; web interface for viewing current network status, notification and problem history, log file, etc.; user defined plug-ins; hierarchical user authorization for access to the web interface

15. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 15 Nagios Status Map

16. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 16 Nagios Alert History

17. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 17 Looking Glass Web-based access for read-only configuration management of network equipments and troubleshooting; User level access authorization; Configuration file viewer; Interfaces status and parameter viewer; IP routing table and/or single IP route viewer; Routing protocols status viewer; Simple debugging tools (ping and traceroute); Router command line interface.

18. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 18 Trouble Ticket System Request Tracker (http://www.bestpractical.com/rt/) Features: web-based interface with user level authentication; multiple queues support (administrative, technical, etc.); interface for ticket submitting and operation via e-mail; granular user access control (requestor, watcher, admin, owner, etc.); SQL database storage system; hierarchical tickets linking system (parent-child relationships); customizable templates for system messages and ticket operation

19. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 19 Trouble Ticket System Preference Settings

20. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 20 Trouble Ticket System Ticket Operation

21. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 21 SmokePing measures latency, latency distribution and packet loss in the network; uses RRDtool to maintain a long-term datastore and to draw graphs; smart alarm system Statistics available through web interface

22. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 22 Network WeatherMap Perl tool that displays the utilization of the network links Statistics available through web interface

23. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 23 RANCID Monitors a router or other device configuration, including software and hardware (cards, serial numbers, etc), using CVS; E-mail reports

24. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 24 Conclusions Virtual NOC framework establishes the responsibilities of each unit involved in the management of a network structure with branches spread over a large geographical area: NME - Network Management Entity SIE – Service Implementer Entity APM - Access Port Managers at each NOC Distributed and centralized strategies and opportunities for outsourcing Most of the network management software tools are available as Open Source The availability of the network for the last four months is very close to 100%!

25. TERENA Networking Conference 2004 - Rhodes, Greece, 7-10 June 2004 25 Questions and Thanks