1. The GrangeNet NOC Greg Wickham

2. Contents Goals Design Features Architecture Implementation Future Conclusion

3. Contents Goals Design Features Architecture Implementation Future Conclusion

4. Goals Central repository where all information about the network is available Monitoring of the network including availability and utilisation Access control to ensure that the right people have access to the right data Leverage existing utilities where practical

5. Contents Goals Design Features Architecture Implementation Future Conclusion

6. Design Features Easy to use –Intuitive Web Framework aids rapid development Modular Construction –Extensible Security from the ground up Unified configuration –Mimimise replication of data

7. Contents Goals Design Features Architecture Implementation Future Conclusion

8. Architecture Log file monitoring Security server Database backend Historical configurations

9. Architecture

10. db Database contains information on: devices; connections; interfaces; as numbers; prefixes; subnets; members

11. Architecture devices db logfiles Devices connected to the network syslog to the NOC

12. Architecture devices logwatcher db logfiles Logwatcher process monitors log files; messages from devices defined in the db are stored in the db

13. Architecture devices logwatcher pancho db configs logfiles Logwatcher notices a configuration change; pancho is used to download the config and store in a RCS

14. Architecture devices logwatcher pancho utilisation db configs logfiles datacollector Datacollector periodically examines SNMP counters on device; stores in RRDTool db

15. Architecture devices logwatcher pancho utilisation db configs logfiles tacasip datacollector Tacacs authentication is used for all Cisco devices; no shared enable secret

16. Architecture devices logwatcher pancho conf report utilisation db configs logfiles tacasip datacollector Configuration change reports are mailed to the noc-manager nightly

17. Architecture devices logwatcher pancho conf report utilisation www db configs logfiles tacasip datacollector Web server is primary interactive portal

18. Architecture devices logwatcher pancho conf report utilisation weathermap www db configs logfiles tacasip datacollector Weathermap renders graphic of total network utilisation

19. Architecture devices logwatcher pancho conf report utilisation weathermap www db configs logfiles tacasip datacollector grapher Grapher renders historical graphs

20. Architecture devices logwatcher pancho conf report utilisation weathermap www Config generator db configs logfiles tacasip datacollector grapher ‘cookie cutter’ configurations are generated

21. Architecture devices logwatcher pancho conf report utilisation weathermap www Config generator db configs logfiles tacasip datacollector grapher

22. Contents Goals Design Features Architecture Implementation Future Conclusion

23. Leverage Open Source Software

24. The Perl Libraries Unified configuration format RCS Object Orientated HTML Dynamic Web Framework

25. Unified configuration format Similar to Apache COORD-RELATIVE edge1.act COORD-SYSTEM polar #polar coords are ( distance, angle ) COORD 130,-45 LABEL TEXT apac RADIUS 20

26. RCS Based on CPAN RCS Added support for ‘-p’ option –Diff into array

27. Object Orientated HTML Much easier to render dynamic HTML Automatic closure Inline back patching my $html = new HTML::OO; $html->table()->tr()->td(‘Hello World’); my $a = $html->a( { href => ‘link’ } ); $a->print(“Click here”);

28. Dynamic Web Framework All web pages are rendered in real time Concept of Modules and Hierarchy Integrated cookie based authentication –Integrated with LDAP Simplifies writing new modules, especially when the new module: –Is required to integrate with existing modules –Needs authentication protection –Can leverage existing library routines

29. The Database Uses PostgreSQL Contains information describing the network –Who is connected? –Where are they connected? –What devices are connected? –What interfaces are on the devices? –Asset management

30. The Data Collector Extracts list of devices to monitor from database MIBS automatically monitored by defined device classes –Automatically tracks hardware For interfaces – interface name is used For others – SNMP index is used (ie: cpu etc)

31. The tacacs server Tacacs used as authentication protocol Backended into AARNet LDAP Stateful protocol - No shared enable password Accounting (console command logging) Supports some NAS features

32. The Logwatcher Monitors logged output from network devices –Logged messages are screen for sanity (bad formatting etc.) –Inserted into the database –Any configuration change triggers a download of the configuration –Store configuration files stripped of ‘secrets’ Stored in RCS

33. The Web Server Uses Apache – Perl embedded interpreter (mod_perl) – mod_ssl / OpenSSL (Security) Mod_perl –Increases Apache memory footprint Flexibility (access internal API) Performance (db access) –Decreases CGI script startup time

34. The Configuration Reporter Every night all registered devices are checked –Are running-config and startup-config the same? –Report any changes made in the last 24 hours –Summary report emailed to noc-manager

35. The weathermapper Goal – At a glance network status Configuration file defines the weathermap –Nodes: Defines nodes and connections –Map: Defines nodes to draw Automatically generates: –Full map (all data rendered as requested) –Quick mapping Background pre-rendered Foreground drawn over background –Meta data for ISMAP clicking

36. The weathermapper Goal – At a glance network status Configuration file defines the weathermap –Nodes: Defines nodes and connections –Map: Defines nodes to draw Automatically generates: –Full map (all data rendered as requested) –Quick mapping Background pre-rendered Foreground drawn over background –Meta data for ISMAP clicking

37. The weathermapper Embedded Nagios status

38. The weathermapper Throughput on link Percentage utilisation indicated by colour

39. The weathermapper Legend for colurs / percentage utilisation

40. The weathermapper Links down

41. The weathermapper Connection status highlight (In progress)

42. The Support Cast Suite of scripts that: –Collect SNMP data –Check interface names against db –Analyse Nagios status (scraping)

43. The Result Live demo http://noc.grangenet.net GrangeNet Survey

44. Contents Goals Design Features Architecture Implementation Future Conclusion

45. The Future Nagios? – Where does it fit –Automatic config generation (from db) –Many similar queries to the same device aren’t efficient More reporting –Link statistics –Bandwidth utilisation Shim layer –Scraping currently used –Improved efficiency if APIs are well defined Ie: Nagios

46. Contents Goals Design Features Architecture Applications Implementation Future Conclusion

47. Perl – Rapid application development Proven to be very useful Centralised data –Describes the entire network –The one source of information

48. Questions? greg.wickham@grangenet.net http://noc.grangenet.net