Open Source Network Monitoring Tools Yasir Iqbal 22-May-2010

In this presentation Introduction What are Network Monitoring Tools Bandwidth Monitoring Techniques/Services Setting up some monitoring Tools Conclusion

Introduction:- Why do we need to monitor and measure Bandwidth Cost of Bandwidth is expensive for developing countries  Bandwidth in developing countries is expensive. In a report for the Partnership for Higher Education in Africa, Mike Jensen calculates that Makerere University pays about $22,000/month for 1.5Mbps/768Kbps (in/out), Eduardo Mondlane pays $10,000/month for 1Mbps/384Kbps, while the University of Ghana pays $10,000/month for 1Mbps/512Kbps. These figures indicate that African universities, outside of South Africa, are paying over $55,000/month for 4Mbps inbound and 2Mbps outbound. These figures are about 100 times more expensive than equivalent prices in North America or Europe.

Cont… To Know if the ISP is providing us with the required bandwidth paid for. To be able to optimize the available bandwidth ◦ 59% of institutions do not monitor or manage bandwidth at all ( Belcher)

Ways to improve network performance Upgrade infrastructure, to install faster, larger, and higher performing systems, lines and facilities. Look for cheaper provider and Increase/upgrade your bandwidth. Alternative approach ◦ is to recognize that ‘bandwidth’ is a valuable institutional resource or asset that needs to be managed, conserved, and shared as effectively as possible.

How do we measure Bandwidth? Network Monitoring Tool

What are Network Monitoring Tools? Allows the administrator to know the health status of the network. It provides information about collected data and the analysis of such raw data with a view to using scarce or limited resources effectively. Uses network probe. Probes let you isolate traffic problems and congestions slowing your network to a crawl.

What can we use the tools for? Identifying unofficial services or servers Monitoring usage and traffic statistics Troubleshooting your network Investigating a security incident Keeping logs of users activities for accountability

Who? What? Where? How? When? Who is accessing your network? ◦students, academics, staff, visitors or others What are they accessing your network for? ◦academic study, social use, business use, illegal use Where are they accessing your network from? ◦internal, external How are they accessing your network? ◦remote user, local Ethernet, WAN, dial-up, Wi-Fi, VPN When did they access your network? ◦today, yesterday, last week, last month…

Network Monitoring Tools Active tools ◦Ping – test connectivity to a host ◦Traceroute – show path to a host ◦MTR – combination of ping + traceroute ◦SNMP collectors (polling) ‏ Passive Tools ◦MRTG ◦Nagios ◦Cacti ◦Ntop ◦Webalizer

Passive Network Monitoring Tools Multi-Router Traffic Grapher Is a tool for monitoring traffic loads on a network link. MRTG generates HTML pages that provide a live, visual representation of the network traffic. It can be used to monitor any SNMP MIB. Limitations ◦It cannot provide information that shows which host or application may be causing a traffic bottleneck. ◦MRTG does not provide information about traffic type or protocol statistics

CONT… TCPdump ◦Uses the packet capture library (libpcap). ◦Prints the headers of packet on a network interface, user analyses network status using this header manually ◦Has many option for capturing raw data, but it does not provide any analysis capability for the captured data.

CONT….. IPTraf ◦IPTraf is a console-based network statistics utility for Linux. It gathers a variety of figures such as TCP connection packet and byte counts, interface statistics and activity indicators, TCP/UDP traffic breakdowns, and LAN station packet and byte count ◦Protocols Recognized ◦IP ◦TCP ◦UDP ◦ICMP ◦IGMP ◦IGP ◦IGRP ◦OSPF ◦ARP ◦RARP

CONT… Webalizer ◦The Webalizer is a fast, free web server log file analysis program. It produces highly detailed, easily configurable usage reports in HTML format, for viewing with a standard web browser. ◦ _ html

Nagios an enterprise-class network and server monitoring system. Useful for: ◦Monitoring of network services. ◦Monitoring of host resources (processor load, disk usage, system logs) ◦Contact notifications when service or host problems occur and get resolved (via , SMS). ◦You can define event handlers that execute when triggered by certain events. (Proactive problem resolution)

OpenNMS Functionalities ◦High performance  A single instance of OpenNMS supports monitoring of a large number of nodes. ◦Automation  OpenNMS minimizes the amount of manual configuration. ◦Rule-based configuration  Flexible rules can be used to specify what services are polled on what devices.

Cacti Similar to MRTG. Based on RRDtool. Offers excellent graphing capabilities. Has extensive templates.

General Description of Cacti 1. Cacti is written as a group of PHP scripts. 2. The key script is “poller.php”, which runs every 5 minutes (by default). It resides in /usr/share/cacti/site. 3. To work poller.php needs to be in /etc/cron.d/cacti like this: MAILTO=root */5 * * * * www-data php /usr/share/cacti/site/poller.php >/dev/null 2>/var/log/cacti/poller-error.log 4. Cacti uses RRDtool to create graphs for each device and data that is collected about that device. You can adjust all of this from within the Cacti web interface. 5. The RRD data is stored in a MySQL database along with descriptions of each device that is monitored. 6. The RRD files are located in /var/lib/cacti/rra.

Advantagess You can measure Availability, Load, Errors and more all with history. –Cacti con view your router and switch interfaces and their traffic, including all error traffic as well. –Cacti can measure drive capacity, CPU load (network h/w and servers) and much more. It can react to conditions and send notifications based on specified ranges. Graphics –Allows you to use all the functionality of rrdgraph to define graphics and automate how they are displayed. –Allows you to organize information in hierarchical tree structures. Data Sources –Permits you to utilize all the functions of rrdcreate and rrdupdate including defining several sources of information for each RRD file.

Advantages cont. Data Collection –Supports SNMP including the use of php-snmp or net- snmp –Data sources can be updated via SNMP o by defining scripts to do this. –An optional component, cactid, implements SNMP routines in C with multi-threading. Important for very large installations, but not tested formally. Templates –You can create templates to reutilize graphics definitions, data and device sources User Management –You can manage users locally or via LDAP and you can assign granular levels of authorization by user or groups of users.

Disadvantages Configuration of Interfaces is Tedious –The first time you add an interfaces, add graphics for each interface and place these graphics correctly on a hierarchical menu requires considerable time and effort. –It’s very important that you keep your Cacti configuration up-to-date with your network. You must either assign someone to do this, or create appropriate scripts and data shares for this purpose. –If you make a configuration error it can be tedious to correct it. But, in reality, for continuous use or large installations it is likely that you will be using scripts and tools to automate the configuration of Cacti.

Setting up Cacti on CentOS 5 Cacti requires that the following software is installed on your system. ◦RRDTool or 1.2.x or greater ◦MySQL 4.1.x or 5.x or greater ◦PHP or greater, 5.x greater highly recommended for advanced features ◦A Web Server e.g. Apache ◦Net-Snmp Mysql, PHP, Apache and SNMP packages are already installed on your machine if not installed through yum utility. yum install mysql-server mysql php-mysql php-pear php-common php-gd php-devel php php-mbstring php-cli php-snmp php-pear- Net-SMTP php-mysql httpd

rrdtool: Installation Install rrdtool manually by downloading the latest version at the following URL SCP the tarball into the /usr/src directory on your linux box. From a command prompt, change into the /usr/src directory, and un-tar the tarball: cd /usr/src tar -xzvf rrdtool tar.gz Change into the newly created directory: cd rrdtool Compile and install RRDTool:./configure make make install

rrdtool: Installation The default installation location is /usr/local/rrdtool-VERSION, so make some symbolic links to the executables: ln -sf /usr/local/rrdtool /bin/rrdtool /usr/bin/rrdtool ln -sf /usr/local/rrdtool /bin/rrdupdate /usr/bin/rrdupdate ln -sf /usr/local/rrdtool /bin/rrdcgi /usr/bin/rrdcgi The RRDTool Perl library simplifies things when using RRDTool from a Perl script, so to compile and install the Perl library for RRDTool: make site-perl-install Create a directory for RRDTool databases, and a directory for the web images which it'll generate: mkdir /var/lib/rrd mkdir /var/www/html/rrdtool

cacti: Installation Extract the distribution tarball. shell> tar xzvf cacti-version.tar.gz Create the MySQL database: shell> mysqladmin --user=root create cacti Import the default cacti database: shell> mysql cacti < cacti.sql Optional: Create a MySQL username and password for Cacti. shell> mysql --user=root mysql mysql> GRANT ALL ON cacti.* TO IDENTIFIED BY 'somepassword'; mysql> flush privileges;

cacti: Installation Edit include/config.php and specify the MySQL user, password and database for your Cacti configuration. $database_default = "cacti"; $database_hostname = "localhost"; $database_username = "cactiuser"; $database_password = "cacti"; Set the appropriate permissions on cacti's directories for graph/log generation. You should execute these commands from inside cacti's directory to change the permissions. shell> chown -R cactiuser rra/ log/ (Enter a valid username for cactiuser, this user will also be used in the next step for data gathering.) Add a line to your /etc/crontab file similar to: */5 * * * * cactiuser php /var/www/html/cacti/poller.php > /dev/null 2>&1

cacti: Installation Now use a web browser and open the following address: You will see the following...

cacti: Installation Press “Next >>”

cacti: Installation Choose “New Install” and press “Next >>” again.

cacti: Installation Your screen should look like this. If it does not ask your instructor for help. Press “Finish” Note! Be sure that “RRDTool 1.2.x” is chosen and not “1.0.x”.

cacti: First Login First time login use: User Name:admin Password:admin

cacti: Password Change Now you must change the admin password. Please use the workshop password.

Add Devices: 1 Management -> Devices -> Add Specify device attributes – Choose a device template and this will ask you for additional information about the device. – You can add additional templates when, or if, you want.

Add Devices: 2

Add Devices: 3 Choose SNMP version 2 for this workshop. At your own location you can use SNMP version 3 if your devices support this. SNMP access is a security issue: -Version 2 is not encrypted -Watch out for globally readable “public” communities -Be careful about who can access r/w communities.

Create Graphics Chose the “Create graphs for this host” Under Graph Templates generally check the top box that chooses all the available graphs to be displayed. Press Create. You can change the default colors, but the predefined definitions generally work well.

Create Graphics: Step 1

Create Graphics: Step 2

View the Graphics Place the new device in its proper location in your tree hierarchy. Building your display hierarchy is your decision. It might make sense to try drawing this out on paper first. – Under Management  Graph Trees select the Default Tree hierarchy (or, create one of your own).

Graphics Tree First, press “Add” if you want a new graphing tree: Second, name your tree, choose the sorting order (the author likes Natural Sorting and press “create”:

Graphics Trees Third, add devices to your new tree: Once you click “Add” you can add “Headers” (separators), graphs or hosts. Now we'll add Hosts to our newly created graph tree:

An Example…

Conclusions Cacti is very flexible due to its use of templates. Once you understand the concepts behind RRDTool, then how Cacti works should be (more or less) intuitive. The visualization hierarchy of devices helps to organize and discover new devices quickly. There are very few to no statistics available about the performance of cactid (volunteers are welcome!). It is not easy to do a rediscover of devices. To add lots of devices requires lots of time and effort. Software such as Netdot, Netdisco, IPPlan, TIPP can help – as well as local scripts that update the Cacti back-end MySQL database directly.

NTop Network probe that shows network activity just like “top”.

Setting up Ntop Download Ntop Using a tar ball tar xpfz ntop tar.gz./configure make make install Installing with RPM is also easy. The package name may vary, but you simply use the command: rpm –uvh ntop-3.0-4mdk.i586.rpm Run ntop (service ntop start) Go to a web browser type

Security Tools Some security tools to consider: ◦NetFilter IP Tables – Firewall ◦WireShark – Protocol analyzer ◦Snort – Intrusion detection ◦Netcat – Feature rich tool. Great for debugging. ◦Nessus – Vulnerability scanner ◦Many many more…

?