Presentation is loading. Please wait.

Presentation is loading. Please wait.

Nagios in the Real World

Similar presentations

Presentation on theme: "Nagios in the Real World"— Presentation transcript:

1 Nagios in the Real World
Dave Williams Technical Architect

2 Agenda

3 Agenda Introduction Example Implementations of Nagios
General Background System Monitoring Background Example Implementations of Nagios UK Customer Examples Datacentre Monitoring with Nagios What is a Datacentre ? Software & Hardware combinations Vision Conclusions

4 Background UK based Working for Bull Mainframe (IBM & Honeywell)
Unix (HP-UX, AIX, Solaris) Network (CASE, 3COM, CISCO) Working for Bull French Computer Manufacturer Mainframes, Unix, HPC, Security, Managed Services

5 Background System Monitoring Open Source Monitoring OpenView Netview
Open Master Open Source Monitoring NetSaint on AIX Nagios

6 Example Implementations

7 Crown Office Procurator Fiscal Service
Responsible for the prosecution of crime in Scotland Investigation of suspicious deaths Complaints against the Police IT Locations in Glasgow & Edinburgh Windows at every Courts of Justice in Scotland AIX / Oracle DB at Glasgow & Edinburgh

8 Crown Office Procurator Fiscal Service
Already used Solarwinds for some network monitoring Strategy demanded AIX based monitoring & reporting In a competitive tender Nagios selected Main success points were – simplicity, ease of customisation Fitted within AIX based distance data replication already in use

9 Crown Office Procurator Fiscal Service
60+ Windows systems monitored for CPU, Disk Space etc 2 AIX servers monitored for CPU, Disk Space etc Two Oracle Instances monitored for performance and DBspace usage All alerts shown on monitor screen and if necessary SMS Text alerts Installed 2005, still working Provides ‘backstop’ to Solarwinds for capacity monitoring on the WAN & LAN.

10 Rother District Council
“Working with the community to improve the overall well-being of the District “ Responsible for Waste Collection, Housing, Planning & Building Control The District covers some 200 square miles and serves a population of around 90,000 inhabitants.

11 Rother District Council
Monitoring 20+ Windows Servers for CPU, Disk Utilsation etc Monitoring numerous disparate Applications Reporting on Availability Monitoring Printer status Unexpected benefits

12 North Yorkshire County Council
Internet Access system for 30,000 pupils Monitoring , internet access, IDS, AV, Webservers Reporting on Availability Monitoring Service Level Indicators Mix of application providers (Scalix, Plesk) Mix of appliance systems – Cisco, Panda, Radware, NetEnforcer, MyFilter

13 North Yorkshire County Council
System Schematic

14 North Yorkshire County Council
Uses NRPE to perform active checks on hosts Multi O/S support Debian RedHat Uses NSCA to accept check results from Windows Via NagiosEventLog

15 North Yorkshire County Council
Scalix running on Redhat Cluster. Checking all processes, cluster state etc. PLESK Web server Checking availability of web sites via test installation Monitoring disk utilsation and processor utilisation AV systems Monitoring availability Checking on AV database Myfilter Monitoring filters running Checking that sufficient filters are available

16 North Yorkshire County Council
Nagios server runs external loopback test every 20 minutes to confirm external reachability. PLESK Web server Straightforward implementation of check_http NetBackup Monitoring that backups have run Checking that enough backup tapes are available Business Availability Define which services constitute a business line 07:00 check – tell support before the customers come on line

17 NYCC - Nagiosgraph Nagiosgraph Uses process_performance _data
Example of Unix load average

18 NYCC – Nagios Monitoring
Scalix System

19 NYCC Alerts sent via email to customers as well as support
Backup notifications via SMS Text Use Nagios Looking Glass for Customer View nagiosgraph used to catch all service performance data Debian & Redhat perfomance metrics Network throughput from LAN switches LDAP response time

20 Datacentre Monitoring with Nagios

21 What is a DataCentre ? A data center (or datacentre) is a facility used to house computer systems and associated components, such as telecommunications and storage systems. It generally includes redundant or backup power supplies, redundant data communications connections, environmental controls and security devices. (Wikipedia)

22 How good is your DataCentre ?
The TIA-942:Data Center Standards Overview describes the requirements for the data centre infrastructure. The simplest is a Tier 1 data centre, which is basically a server room, following basic guidelines for the installation of computer systems. The most stringent level is a Tier 4 data centre, which is designed to host mission critical computer systems, with fully redundant subsystems and compartmentalized security zones controlled by biometric access controls methods . (Wikipedia)

23 What is a DataCentre ? Tier 1 Requirements Tier 2 Requirements
Single non-redundant distribution path serving the IT equipment Non-redundant capacity components Basic site infrastructure guaranteeing % availability Tier 2 Requirements Fulfills all Tier 1 requirements Redundant site infrastructure capacity components guaranteeing % availability Tier 3 Requirements Fulfills all Tier 1 and Tier 2 requirements Multiple independent distribution paths serving the IT equipment All IT equipment must be dual-powered and fully compatible with the topology of a site's architecture Concurrently maintainable site infrastructure guaranteeing % availability Tier 4 Requirements Fulfills all Tier 1, Tier 2 and Tier 3 requirements All cooling equipment is independently dual-powered, including chillers and heating, ventilating and air-conditioning (HVAC) systems Fault-tolerant site infrastructure with electrical power storage and distribution facilities guaranteeing % availability ©Uptime Institute uptime = 27 minutes a year = 1.6 hours downtime per year = 22 hours downtime a year = 28.8 hours downtime

24 What is a Green DataCentre ?
The most commonly used metric to determine the energy efficiency of a data centre is power usage effectiveness, or PUE. This simple ratio is the total power entering the data centre divided by the power used by the IT equipment. PUE = Total facility Power / IT Equipment Power Power used by support equipment, often referred to as overhead load, mainly consists of cooling systems, power delivery, and other facility infrastructure like lighting. The average data centre in the US has a PUE of 2.0, meaning that the facility uses one Watt of overhead power for every Watt delivered to IT equipment. State-of-the-art data centre energy efficiency is estimated to be roughly 1.2. = 28.8 hours downtime DCiE = IT Equipment Power / Total Facility Power Data Centre infrastructure Efficiency

25 Bull Datacentre BC1 ? New datacentre build on an already existing site
Design criteria PUE 1.6 Easily expanded on demand Tier 3

26 Bull UK Datacentre BC1 What do you get for £1.2m ?

27 Bull UK Datacentre BC1 New Mains Incomer 1.2Mw Generator
Took feed from 11Kv ring Had to build own substation 1.2Mw Generator Required 8000 litre fuel tank Switchgear to automatically start generator if mains incomer fails (10-45 seconds) 3 x Ambient CRAC Units Cooling via external temperature differential N+1 configuration Hot Aisle Containment In-Line UPS UPS only required to keep IT equipment running until generator fires up Uses space in Cab rows, easily scalable according to load BC1 Latitude 53 degrees – good for ambient cooling! Fuel for 48 hours running time

28 Bull UK Datacentre BC1 - Monitoring
Physical Environment APC Netbotz Devices Translate inputs from sensors Humidity, Temperature, Dew Point SEAL I/O Dry Contact Voltage indicators For CRAC, FM200, Generator, UPS Electrical Efficiency PowerLogic ION software reads from power meters Power meter on every Distribution Board Real-time calculation of PUE Power Distribution Every PDU strip (2 per Cab) monitored for power consumption & problems A number of PDU strips also have remote control down to socket level Management Network LAN infrastructure required to support the Datacentre Servers required to support the datacentre External alert mechanisms BC1 Latitude 53 degrees – good for ambient cooling! Fuel for 48 hours running time

29 Bull UK Datacentre BC1 What does Netbotz look like ?

30 Bull UK Datacentre BC1 What does SeaLevel look like ?

31 Bull UK Datacentre BC1 What does ION look like ?

32 Bull UK Datacentre BC1 What does a metered PDU look like ?

33 Bull UK Datacentre BC1 What does a managed PDU look like ?

34 Bull UK Datacentre BC1 Nagios Map

35 Bull UK Datacentre BC1 Nagios Host Groups

36 Bull UK Datacentre BC1 Do things go wrong - yes

37 Bull UK Datacentre BC1 Do things go wrong - yes & no

38 Datacentre Monitoring Schematic

39 Nagios Products in use Nagios Core Nagios Looking Glass Nagvis EventDB
NRPE NSCA Nagios Looking Glass Nagvis EventDB SNMPTT Nagmap NDO

40 Other Open Source Products in use
Nedi Arpwatch PSAD SMS-Client Bacula Confluence (Wiki) i-doit (ITIL CMDB) MRTG Routers2cgi

41 BC1 Datacentre Monitoring Elements
Nagios Core Normal install with direct polling of devices Only looking at Datacentre Nagios Display System Central reporting Nagios Absorbs updates from other Nagios instances Information Display Normal system with 5 heads Nagios Customer System Running on an appliance connected to Customer network Sends data via encrypted secured link to Display System Backup System Use tape library Hosts CMDB & WiKi

42 BC1 Datacentre Nagios Core
Hardware Platform - Intel O/S Centos 5 Xeon 2.8Ghz , 8Gb memory, 72GB RAID-1 disk Nagios 3.2.0 Built from source tarball Nagios Plugins Installed from RPM

43 BC1 Datacentre Nagios Display System
Hardware Platform - Intel O/S Fedora Core 9 P4 2.8Ghz , 2.5Gb memory, 76GB RAID-1 disk Nvidia dual monitor display Card – DVI interfaces Nagios 3.0.6 Built from source tarball Nagios Plugins Installed from RPM

44 BC1 Datacentre Normal Display System
Hardware Platform - AMD O/S Centos 5 Athlon 1.2Ghz , 1.0 Gb memory, 3GB disk Matrox G200 Quad Head Runs console displays – http/RDP/ssh

45 BC1 Datacentre Customer System
Hardware Platform – Motion Tablet O/S Ubuntu LTS Pentium M 1.5Ghz , 0.5 Gb memory, 30GB disk Touch Screen tablet system Nagios 3.2.3 Built from tarball Nagios Plugins Nagios NSCA Sends status (encrypted) to central reporting system

46 BC1 Datacentre Backup System
Hardware Platform – Intel O/S Centos 5 Xeon 3.06Ghz , 2.0 Gb memory, 108GB disk Uses Bacula 5.0.3 Controls SDLT 20 slot tape library Backs up all Datacentre Infrastructure Windows Centos Ubuntu

47 Conclusions

48 Conclusions Strategic Overall Design
Know what you need to monitor Know who needs to be told Expect to throw the first version away Only when you have fully engineered the solution will you understand all of the issues Keep a record of design decisions You will have to make it pretty for management Accept that an attractive display will be required Reporting will become key It must be reliable Make backups Consider clustering & recovery options

49 & Hints

50 Hints & Experience Separate Display systems from Monitoring systems
If you are tracking 10,000’s of services you don’t want processor heavy graphics as well Escalation & Alerting take time Firstly to get right with your organisation Secondly to actually physically do ! Suppliers go out of their way to make it difficult Don’t give in – there is always a way to get Nagios involved Screen scrape, , telnet,RS232 are all possible SNMP is your friend When in doubt use SNMP to help you out SNMP V3 with AES cypher is suitably secure for most implementations





Download ppt "Nagios in the Real World"

Similar presentations

Ads by Google