1. Joe McManus joe@cert.org
SiLK Installation
A Walk-through
Joe McManus
© 2007 Carnegie Mellon University

2. © 2007 Carnegie Mellon University
Topics
Topics Covered in this Training
What SiLK is and is not
SiLK on a Box
SiLK with remote flow collection
Building SiLK RPMs
Monitoring SiLK Processes
Basic SiLK Queries
Useful queries for an administrator
© 2007 Carnegie Mellon University

3. Introduction to SiLK System for internet Level Knowledge
Data collection
Data analysis
NOT:
Intrusion detection/prevention system
Audit tools for your network
Automated report generator

4. Collection & Analysis Infrastructure
SiLK stores the data and allows access to the data for analysis. YAF turns packets into flows for SiLK over IPFIX. SiLK can take input from IPFIX or Netflow. Netflow is convenient as it implemented on many routers and requires no additional hardware.

5. Collection & Analysis Infrastructure

6. © 2007 Carnegie Mellon University
SiLK on a Box Standalone flow collection
© 2007 Carnegie Mellon University

7. Standalone Collection and Analysis
We will now go over installing SiLK and YAF to create a stand alone flow collection and analysis box. We will assume you are on a linux server with root access.

8. Downloading SiLK
SiLK can be downloaded from: YAF is available from: Fixbuf is available from:

9. Downloading (continued…)
Log in to your Linux server and use wget to download the software.
tmp]#cd /tmp
tmp]# wget \
tmp]# wget \
tmp]# wget \

10. Build fixbuf Fixbuf is required to have YAF talk to SiLK over IPFIX.
tmp]# tar -zxvf libfixbuf tar.gz
tmp]# cd libfixbuf-0.8.0
libfixbuf-0.8.0]# ./configure && make && make install

11. Errors
checking for cc... no
checking for cl... no
configure: error: no acceptable C compiler found in $PATH
We need to make sure the system has the required packages to build the suite.

12. YUM We can use YUM to install the missing C compiler.
libfixbuf-0.8.0]# yum install gcc
Try building Fixbuf again.
Which brings us to the next error:
checking whether we are using the GNU C++ compiler... no
checking whether g++ accepts -g... no
checking dependency style of g++... none

13. YUM
If you do not know which package you are looking for YUM can help with this.
libfixbuf-0.8.0]# yum search c++
libfixbuf-0.8.0]# yum install gcc-c++

14. RPM Confusion This error is misleading.
libfixbuf-0.8.0]# ./configure
checking for GLIB - version >= no
*** Could not run GLIB test program, checking why...
*** The test program failed to compile or link. See the file config.log for the
*** exact error that occured. This usually means GLIB is incorrectly installed.
configure: error: Cannot find a suitable glib2 (>= 2.4.7)
This error is misleading.

15. RPM Confusion Wuh?!? [root@silk ~]# rpm -qa | grep glib2
libfixbuf-0.8.0]# yum install glib2
Loaded plugins: refresh-packagekit
Setting up Install Process
Parsing package install arguments
Package glib fc9.i386 already installed and latest version
Nothing to do
~]# rpm -qa | grep glib2
glib fc9.i386
Wuh?!?

16. RPM Confusion What is actually missing is the glib2-devel rpm.
~]# yum install glib2-devel
You may have to install the libpcap-devel rpm as well. This is dependent on which install of linux you have done.

17. Fixbuf install
libfixbuf-0.8.0]# ./configure libfixbuf-0.8.0]# make libfixbuf-0.8.0]# make install libfixbuf-0.8.0]# ls /usr/local/lib libfixbuf so.8 libfixbuf.a libfixbuf.so libfixbuf so libfixbuf.la pkgconfig

18. YAF Install YAF is our tool for flow collection, there are others.
libfixbuf-0.8.0]# cd ..
tmp]# tar -zxvf yaf tar.gz
tmp]# cd yaf-1.0.0
checking for libfixbuf >= configure: error: Cannot find a suitable libfixbuf (>= 0.7.2): Package libfixbuf was not found in the pkg-config search path.
Perhaps you should add the directory containing `libfixbuf.pc'
to the PKG_CONFIG_PATH environment variable
No package 'libfixbuf' found

19. YAF Install Didn’t we just install fixbuf?
-Yes, but pkgconfig does not know to look in /usr/local.
yaf-1.0.0]# export \ PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
yaf-1.0.0]# ./configure
yaf-1.0.0]# make
yaf-1.0.0]# make install

20. SiLK Install Uncompress the SiLK tarball.
yaf-1.0.0]# cd ..
tmp]# tar -zxvf silk tar.gz
tmp]# cd silk-1.1.3
We will build in support for PySiLK.
silk-1.1.3]# yum install python-devel

21. SiLK Install (cont…)
silk-1.1.3]# ./configure \ --with-libfixbuf=/usr/local/lib/pkgconfig/ \ --with-python SiLK creates a great summary.

22. Summary
* Configured package: SiLK * Host type: i686-pc-linux-gnu * Source files ($top_srcdir): . * Install directory: /usr/local * Root of packed data tree: /data * Packing logic: via run-time plugin * Timezone support: UTC * Default compression method: SK_COMPMETHOD_NONE * IPv6 support: NO * IPFIX collection support: YES (-pthread -L/usr/local/lib -lfixbuf -lgthread-2.0-lrt -lglib-2.0) * Transport encryption support: NO (gnutls not found) * IPA support: NO * LIBPCAP support: YES (-lpcap) * Python support: YES (-L/usr/kerberos/lib -Xlinker -export-dynamic -ld -lutil -lm -L/usr/lib -lpython2.5 -pthread) *Python package destination: /usr/lib/python2.5/site-packages * Build analysis tools: YES * Build packing tools: YES * Compiler (CC): gcc * Compiler flags (CFLAGS): -I$(top_srcdir)/src/include -DNDEBUG -D_GNU_SOURCE=1 -D_FILE_OFFSET_BITS=64 -O3 -fno-strict-aliasing -Wall -W -Wmissing-prototypes -Wformat=2 -Wdeclaration-after-statement * Linker flags (LDFLAGS): * Libraries (LIBS): -ldl -lm

23. SiLK Install (cont…) [root@silk silk-1.1.3]# make
silk-1.1.3]# make install
silk-1.1.3]# rwfilter --version
rwfilter: part of SiLK 1.1.3; configuration settings:
* Root of packed data tree: /data
* Packing logic: Run-time plug-in
* Timezone support: UTC
* Available compression methods: none [default]
* IPv6 support: no
* IPFIX collection support: yes
* Transport encryption: no
* PySiLK support: /usr/lib/python2.5/site-packages
* Enable assert(): no

24. SiLK Configuration
We need to create a place to store the flow data. The default for is /data. ~]# mkdir /data We will need to create two configuration files for SiLK, sensor.conf and silk.conf

25. silk.conf
A few sample silk.conf files are included with the distribution. They are installed under /usr/local/share/silk.
We would want to use twoway-silk.conf
/data]# cp \ /usr/local/share/silk/twoway -silk.conf .

26. silk.conf [root@silk data]# vi /data/silk.conf #Define Sensors
sensor 0 localhost
class all     sensors localhost end class
version 1

27. SiLK Configuration
class all type 0 in in type 1 out out type 2 inweb iw type 3 outweb ow type 4 innull innull type 5 outnull outnull type 6 int2int int2int type 7 ext2ext ext2ext type 8 inicmp inicmp type 9 outicmp outicmp type 10 other other default-types in inweb inicmp end class

28. silk.conf storage heirarchy

29. silk.conf storage hierarchy.
I find it good practice to have silk store its data under /data/SENSOR-NAME/type. The benefit of storing data this way is that you can use unix groups to control access to flow data on each sensor. i.e. /data/engineering /data/sales /data/logistics

30. silk.conf %N= Sensor Name %T=Type In/Out/int2int… %Y=Year %m=month
# The default path format from SILK_DATA_ROOTDIR path-format "%N/%T/%Y/%m/%d/%x“
%N= Sensor Name
%T=Type In/Out/int2int…
%Y=Year
%m=month
%d=data
%x=flowtype-sensor_YearMonthDay.Hour

31. silk.conf
# The plug-in to load to get the packing logic to use in rwflowpack. # The --packing-logic switch to rwflowpack will override this value. # If SiLK was configured with hard-coded packing logic, this value is # ignored. # The plug-in to load to get the packing logic to use in rwflowpack. # The --packing-logic switch to rwflowpack will override this value. # If SiLK was configured with hard-coded packing logic, this value is # ignored. packing-logic "packlogic-twoway.so“

32. sensor.conf
We will now have to edit the sensor.conf file. data]# vi /data/sensor.conf probe localhost ipfix listen-on-port protocol tcp accept-from-host end probe

33. sensor.conf
sensor localhost     ipfix-probes localhost     internal-ipblock /24     external-ipblock remainder end sensor

34. rwflowpack
rwflowpack is the daemon that will listen for traffic from YAF.
A sample file is included, which we will customize.
cp \ /usr/local/share/silk/etc/rwflowpack.conf \ /usr/local/etc/.
data]# vi /usr/local/etc/rwflowpack.conf

35. rwflowpack.conf
Change the following values: ENABLED=yes SENSOR_CONFIG=/data/sensor.conf SITE_CONFIG=/data/silk.conf LOG_TYPE=legacy LOG_DIR=/var/log CREATE_DIRECTORIES=yes COMPRESSION_TYPE=best

36. rwflowpack.conf
To make rwflowpack start on boot we can add it using chkconfig. SiLK includes sample startup scripts with the distribution.
data]# cp /usr/local/share/silk/etc/init.d/rwflowpack /etc/init.d
data]# chkconfig rwflowpack on
data]# chkconfig --list rwflowpack
rwflowpack :off 1:off 2:on 3:on 4:on 5:on 6:off

37. Test YAF Test that YAF is able to listen on the interface.
data]# yaf --live=pcap --in=eth0 --out=- -v | yafscii
[ :06:10] yaf starting
[ :06:10] running as root in --live mode, but not dropping privilege

38. Test YAF Let that run for a minute. Then enter CTRL-C.
C :07: tcp :1005 => : b24967:0b0cbdd9 AF/A:AF/0 (2/104 <-> 1/52) rtt 0 ms
[ :07:42] Processed 57 packets into 9 flows:
[ :07:42] Mean flow rate 0.18/s.
[ :07:42] Mean packet rate 1.11/s.
[ :07:42] Virtual bandwidth Mbps.
[ :07:42] Maximum flow table size 8.
[ :07:42] 6 flush events.
[ :07:42] Rejected 1 out-of-sequence packets.
[ :07:42] Assembled 0 fragments into 0 packets:
[ :07:42] Expired 0 incomplete fragmented packets.
[ :07:42] Maximum fragment table size 0.
[ :07:42] Rejected 44 packets during decode:
[ :07:42] 44 due to unsupported/rejected packet type:
[ :07:42] unsupported/rejected Layer 3 headers.
[ :07:42] yaf terminating

39. IPTables You must open the firewall for YAF to connect to rwflowpack.
data]# iptables -I INPUT \
-s p tcp -m tcp \
--dport j ACCEPT
data]# service iptables save
iptables: Saving firewall rules to /etc/sysconfig/iptables:[ OK ]

40. IPTables We can check to see if the rule has been added correctly.
data]# service iptables status
Table: filter
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT tcp / tcp dpt:18001 ….

41. Starting YAF
data]# yaf --silk --ipfix=tcp --live=pcap --in=eth0 --out= ipfix-port=18001 &

42. Starting YAF The silk option tells YAF to format the output for silk.
data]# yaf --silk --ipfix=tcp --live=pcap --in=eth0 --out= ipfix-port=18001 &
The silk option tells YAF to format the output for silk.

43. Starting YAF The ipfix=tcp option tells YAF use ipfix over tcp.
data]# yaf --silk --ipfix=tcp --live=pcap --in=eth0 --out= ipfix-port=18001 &
The ipfix=tcp option tells YAF use ipfix over tcp.
The ipfix-port=18001 option specifies which port.

44. Starting YAF
data]# yaf --silk --ipfix=tcp --live=pcap --in=eth0 --out= ipfix-port=18001 &
The live=pcap option tells YAF the capture device uses pcap.
The only other option is dag for use with Endace Dag cards.

45. Starting YAF The in=eth0 option tells YAF which input device to use.
data]# yaf --silk --ipfix=tcp --live=pcap --in=eth0 --out= ipfix-port=18001 &
The in=eth0 option tells YAF which input device to use.
If you are using a dag card, the option would be dag0.

46. Starting YAF
data]# yaf --silk --ipfix=tcp --live=pcap --in=eth0 --out= ipfix-port=18001 &
The out= option tells YAF where to send its output.

47. Checking on YAF
You can check the status of YAF with the following command:
data]# kill -SIGUSR1 `pgrep yaf`
[ :53:29] Rejected 1 out-of-sequence packets.

48. Checking on The Collection Process
When you first start yaf one of the first places to look is in the log file under /var/log.
data]# tail /var/log/rwflowpack log
Jan 8 16:53:40 silk rwflowpack[10605]: Flushing files after 120 seconds.
Jan 8 16:54:20 silk rwflowpack[10605]: Opening new output file /data/localhost/ext2ext/2009/01/08/ext2ext-localhost_

49. Checking on The Collection Process
Did yaf start correctly?
data]# ps -ef | grep yaf
root :52 pts/ :00:00 yaf --silk --ipfix=tcp --live=pcap --in=eth0 --out= ipfix-port=18001

50. Checking on The Collection Process
If not can yaf connect to rwflowpack? data]# telnet localhost Trying Connected to localhost. Escape character is '^]'. Is rwflowpack running? data]# ps -ef | grep yaf

51. Try it out
Lets try out a simple rwfilter command to see if it is working:
data]# rwfilter --type=all --proto=0- --pass=stdout | rwcut | head
sIP| dIP|sPort|dPort|pro| packets| bytes| flags| sTime| dur| eTime| sensor|
| |43393| 22| 6| | |FSRPA |2009/01/08T21:53:25.789| |2009/01/08T21:53: localhost|
| | 22|43393| 6| | | S PA |2009/01/08T21:53:25.789| |2009/01/08T21:53:25.812|localhost|
| | 22|43393| 6| | |F A |2009/01/08T21:53:25.812| |2009/01/08T21:53:25.812|localhost|
| |43393| 22| 6| | | R |2009/01/08T21:53:25.812| |2009/01/08T21:53:25.812|localhost|

52. © 2007 Carnegie Mellon University
Routers Using routers as a data feed
© 2007 Carnegie Mellon University

53. Receiving data from a router
Another common way of receiving data is from a router on a mirror port or span port. In Cisco IOS this is called a span port and if very easy to configure.

54. Span Port IOS Example: interface FastEthernet0/24
port monitor FastEthernet0/1
port monitor FastEthernet0/2
…..
port monitor FastEthernet0/23
port monitor VLAN1

55. YAF & Span port
You would set up sensor.conf with the same options as in the previous example. Although you would want to have a second interface on the box listening for it set up in promiscuous mode with arp disabled as well. Yaf would also start with the same options as before, but pointing to your second nic.

56. SiLK & Netflow
Silk can take netflow directly from a netflow compatible router. You would have to open your firewall to accept UDP on the port you decide to use and make changes to sensor.conf as show in the next slide.

57. SiLK & Netflow
probe S2 netflow-v5 listen-on-port 9901 protocol udp accept-from-host end probe sensor S2 netflow-v5-probes S2 internal-ipblock /16 external-ipblock remainder end sensor

58. YAF & DAG Cards
YAF is capable of including support for Endace DAG Cards.
It is it as simple as adding -with-dag option to ./configure.
When YAF is started you would use the following options:
data]# yaf--silk --ipfix=tcp --live=dag --in=dag0 --out= ipfix-port=18001

59. © 2007 Carnegie Mellon University
Working With Taps Tunnelling
© 2007 Carnegie Mellon University

60. Working With Taps

61. Tunneling
YAF can only listen on one interface at a time. If we are using YAF with a tap one interface will receive inbound and one would receive outbound traffic. For some programs like snort this is useful. But with SiLK it classifies traffic by CIDR block in sensor.conf. We can use port bonding on Linux to solve this issue.

62. Tunneling
Fedora 9 works fine with the Trendnet usb ethernet adapter available at Radio Shack.

63. Tunneling
Plug in both adapters and run dmesg looking for the mac address to use in the ifcfg-ethX file.
~]# dmesg | tail -20
usb 1-8: new high speed USB device using ehci_hcd and address 5
usb 1-8: configuration #1 chosen from 1 choice
usb 1-8: New USB device found, idVendor=0b95, idProduct=7720
usb 1-8: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-8: Product: AX88772
eth1: register 'asix' at usb-0000:00:1d.7-8, ASIX AX88772 USB 2.0 Ethernet, 00:50:b6:04:47:11
usbcore: registered new interface driver asix
usb 1-7: new high speed USB device using ehci_hcd and address 6
usb 1-7: configuration #1 chosen from 1 choice
eth2: register 'asix' at usb-0000:00:1d.7-7, ASIX AX88772 USB 2.0 Ethernet, 00:50:b6:04:46:fc
usb 1-7: New USB device found, idVendor=0b95, idProduct=7720
usb 1-7: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 1-7: Product: AX88772

64. Tunneling DEVICE=bond0 BOOTPROTO=static ONBOOT=yes PROMISC=yes
~]# cd /etc/sysconfig/network-scripts/
network-scripts]# vi ifcfg-bond0
DEVICE=bond0
BOOTPROTO=static
ONBOOT=yes
PROMISC=yes
USERCTL=no
PEERDNS=no

65. Tunneling
network-scripts]# vi ifcfg-eth1 DEVICE=eth1 HWADDR=00:50:b6:04:67:11 BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes

66. Tunneling
network-scripts]# vi ifcfg-eth2 DEVICE=eth2 HWADDR=00:50:b6:04:67:12 BOOTPROTO=none ONBOOT=yes MASTER=bond0 SLAVE=yes

67. Tunneling You need to load the bonding module:
~]# vi /etc/modprobe.conf
alias bond0 bonding
Next you can start the bonded interface:
network-scripts]# ifup bond0

68. Tunneling
On some systems the PROMISC=yes option in ifcfg-bond0 is ignored. If it is then editing /etc/rc.d/rc.local and adding it there can help: data]# vi /etc/rc.d/rc.local /sbin/ifconfig bond0 promisc –arp up

69. © 2007 Carnegie Mellon University
RPMs Standardizing installs
© 2007 Carnegie Mellon University

70. RPMs
RPMs make management of software easier. In some situations you may have remote sensors and may not want to include compilers on them, RPMs make installing software easier. Our software generates .spec files for RPMs during the ./configure process.

71. RPM You first need to create a .rpmmacros file.
libfixbuf-0.8.0]# vi ~/.rpmmacros
%packager joe mcmanus
%vendor cert.org
%_topdir /tmp/rpms

72. RPM Create the build heirarchy. [root@silk tmp]# mkdir /tmp/rpms
tmp]# mkdir /tmp/rpms/SPECS
tmp]# mkdir /tmp/rpms/SOURCES
tmp]# mkdir /tmp/rpms/SRPMS
tmp]# mkdir /tmp/rpms/RPMS
tmp]# mkdir /tmp/rpms/BUILD

73. RPM Install the rpms to build rpms. Clean up our previous installs
tmp]# yum install rpm-build
Clean up our previous installs
tmp]# cd libfixbuf-0.8.0
libfixbuf-0.8.0]# make uninstall
libfixbuf-0.8.0]# cd ..
tmp]# cd yaf-1.0.0; make uninstall; cd ..
tmp]# cd silk-1.1.3; make uninstall; cd ..

74. RPM-fixbuf Prepare the fixbuf rpm [root@silk tmp]# cd libfixbuf-0.8.0
libfixbuf-0.8.0]# make clean
libfixbuf-0.8.0]# ./configure
libfixbuf-0.8.0]# cp libfixbuf.spec /tmp/rpms/SPECS/
libfixbuf-0.8.0]# cd /tmp/rpms/SPECS/
SPECS]# cp /tmp/libfixbuf tar.gz /tmp/rpms/SOURCES/
SPECS]# rpmbuild -ba libfixbuf.spec

75. RPM-fixbuf Now that you have built the rpms, install them.
SPECS]# rpm -iv ../RPMS/i386/libfixbuf-*.rpm
Preparing packages for installation...
libfixbuf
libfixbuf-devel-0.8.0
We have to install the devel package to build the yaf package.
You would not need devel in production.

76. RPM -YAF
SPECS]# cp /tmp/yaf tar.gz /tmp/rpms/SOURCES/ SPECS]# cp /tmp/yaf-1.0.0/yaf.spec SPECS]# rpmbuild -ba yaf.spec

77. RPM-YAF
Install it.
SPECS]# rpm -ivh ../RPMS/i386/yaf i386.rpm
Preparing ########################################### [100%]
file /usr/lib/libltdl.so.3 from install of yaf i386 conflicts with file from package libtool-ltdl fc9.i386
YAF uses autoconf to build and it can’t find the libtool headers so it tries to install its own version of libtool.

78. RPM-YAF
What we need to install is libtool-ltdl-devel and edit yaf.spec.
SPECS]# vi yaf.spec
<go to the %build section append this to the end>
--enable-ltdl-install=no
SPECS]# rpmbuild -ba yaf.spec
SPECS]# rpm -iv ../RPMS/i386/yaf i386.rpm

79. RPM-SiLK
We now need to build the SiLK rpms. To do this we must first install flex and bison. SPECS]# yum install -y flex bison SPECS]# cp /tmp/silk-1.1.3/silk.spec . SPECS]# cp /tmp/silk tar.gz ../SOURCES/ SPECS]# rpmbuild -ba silk.spec

80. RPM-SiLK
Silk builds many rpms. This gives you the option to just install parts of SiLK on one host, i.e. just the collector, and the full analysis suite on another.
i386]# rpm -ivh silk-common i386.rpm \
silk-analysis i386.rpm \
silk-rwflowpack i386.rpm

81. RPM-SiLK There are a few differences with the SiLK rpm install.
Installs rwflowpack.conf /etc/sysconfig/
I create a link /etc/rwflowpack.conf
i386]# ln -s /etc/sysconfig/rwflowpack.conf /etc/rwflowpack.conf
It copies rwflowpack to init.d for you.

82. RPM-Yaf
YAF comes with a sample startup script called Airdaemon. I use my own. ~]# vi /etc/init.d/yaf <See sample on wiki> ~]# chkconfig --add yaf ~]# service yaf start

83. © 2007 Carnegie Mellon University
Monitoring
© 2007 Carnegie Mellon University

84. rwfilter Top twenty talkers.
~]$rwfilter --proto=0- --type=all --pass=stdout | rwstats --top --flows --count 10 --sip
INPUT SIZE: 2062 records for 66 unique keys
SOURCE IP Key: Top 10 flow counts
sIP| Records|%_of_total| cumul_%|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |

85. rwfilter Top 20 Machines by bytes and source ip.
~]$ rwfilter --sensor=localhost --type=all --proto=0- --pass=stdout | rwstats --top --bytes --count=20 --sip
sIP| Bytes|%_of_total| cumul_%|
| | | |
| | | |
| | | |
| | | |
| | | |

86. rwfilter Top 20 machines by destination/incoming.
~]$ rwfilter --sensor=localhost --type=all --proto=0- --pass=stdout | rwstats --top --bytes --count=20 --dip
INPUT SIZE: 9195 records for 255 unique keys
DESTINATION IP Key: Top 20 byte counts
dIP| Bytes|%_of_total| cumul_%|
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |
| | | |

87. rwfilter
~]$ rwfilter --sensor=localhost --type=all --proto=0- --pass=stdout --daddress= | rwsort --fields=bytes | rwsort --fields=bytes| rwcut --fields=sip,dip,dport,sport,dur| tail
| |62168| 139| |
| |62418| 139| |
| |62419| 139| |
| |62420| 139| |
| |62404| 139| |
| |62403| 139| |
| |62402| 139| |
| |56022| 53| |
| |62086| 139| |
| |62086| 139| |

88. rwfilter
As an administrator you may want to check for scanners or set a baseline for expected amount of traffic for each day and check to see if your total traffic is % above on a day and look for the root cause – youtube netflix streaming “slashdotting”

89. Monitoring A simple ps type script can monitor yaf. #!/bin/bash
proccount=“1”
proc=“yaf”
running=`ps –ef | grep yaf | grep –c –v grep >/dev/null`
If [ “$running” –lt “$proccount” ]
then
mail –s”$proc not running `date`” $mailto <<EOF
Less than $proccount $proc running on $HOSTNAME at ‘date`
Message created by $0
EOF fi

90. Monitoring
The same script can be used for rwflowpack. But what about if it is running and a link in the chain has broken and no data is being stored? We can use rwfilter to look for flows created in the last 30 minute period, we could tail the rwflowpack.log and look for new files being written.

91. Monitoring
This example will use rwfilter to check for flows created in the last hour.
now=`date –utc +%Y/%m/%d:%H`
for loop in `ls -F /data/ | grep \/ | sed ‘s/\///’` do
flowcount=` rwfilter –sensor=$loop \ –start-date=$now \
--type=all –proto=0- \
| rwcut | wc –l `
If [ “$flowcount “ –eq “0” ]
then
mail –s”Error: $loop has no records” fi

92. © 2007 Carnegie Mellon University
Wrap up
© 2007 Carnegie Mellon University

93. Going Forward
This system we set up works nicely. But it can be improved. We can run yaf, rwflowcap and rwsender on the sensor and then rwreceiver and rwflowpack on the packing machine. Now if yaf cannot connect to rwreciever it stops, data would be lost.

94. Questions?
Any questions? with any questions you have.