1. CIS 191 - Lesson 12 System Monitoring 1

2. CIS 191 - Lesson 12 System Monitoring Monitoring Log Files /var/log ‒ Can be used as indication of systematic degradation log rotation ‒ logrotate ‒ /etc/logrotate.conf 2

3. CIS 191 - Lesson 12 System Monitoring Many important logs (Red Hat family) Kernel and system boot messages dmesg boot.log (broken – see bugzilla)bugzilla Security and authorization messages secure, btmp, wtmp, lastlog, audit, … System module messages messages (a good catch-all log), cron, maillog, … Key Log File Locations: /var/log directory /etc/syslog.conf 3

4. CIS 191 - Lesson 12 Logging The syslog daemon, controlled by /etc/syslog.conf, is a central clearing house for handling all the log messages sent by various system programs The klogd daemon handles kernel log messages. klogd does not have a configuration file and is controlled by command line switches. [root@opus ~]# ps -e | grep log 2152 ? 00:00:07 syslogd 2155 ? 00:00:00 klogd [root@opus ~]# 4

5. CIS 191 - Lesson 12 System Monitoring Most log files are ascii text messages: ASCII English text messages.1: ASCII text messages.2: ASCII English text messages.3: ASCII English text ppp: directory prelink: directory rpmpkgs: ASCII text rpmpkgs.1: ASCII text rpmpkgs.2: ASCII text rpmpkgs.3: ASCII text samba: directory scrollkeeper.log: ASCII text secure: ASCII text secure.1: empty secure.2: ASCII English text secure.3: ASCII text spooler: empty spooler.1: empty spooler.2: empty spooler.3: empty tallylog: empty vbox: directory wtmp: data wtmp.1: data Xorg.0.log: ASCII English text Xorg.0.log.old: ASCII English text yum.log: ASCII text [root@benji log]# 5

6. CIS 191 - Lesson 12 System Monitoring [root@opus ~]# ls -l /var/log total 153572 -rw-r----- 1 root root 3665 Nov 11 13:36 acpid -rw------- 1 root root 527440 Jun 16 15:47 anaconda.log -rw------- 1 root root 22282 Jun 16 15:47 anaconda.syslog -rw------- 1 root root 58040 Jun 16 15:47 anaconda.xlog drwxr-x--- 2 root root 4096 Nov 24 02:03 audit -rw------- 1 root root 0 Nov 23 04:02 boot.log -rw------- 1 root root 0 Nov 16 04:02 boot.log.1 -rw------- 1 root root 0 Nov 9 04:02 boot.log.2 -rw------- 1 root root 0 Nov 2 04:02 boot.log.3 -rw------- 1 root root 0 Oct 26 04:03 boot.log.4 -rw------- 1 root utmp 136987008 Nov 29 15:16 btmp drwxr-xr-x 2 root root 4096 Jun 28 2007 conman drwxr-xr-x 2 root root 4096 Jun 28 2007 conman.old -rw------- 1 root root 12817 Nov 29 16:01 cron -rw------- 1 root root 13860 Nov 23 04:02 cron.1 -rw------- 1 root root 13706 Nov 16 04:02 cron.2 -rw------- 1 root root 13843 Nov 9 04:02 cron.3 -rw------- 1 root root 14117 Nov 2 04:02 cron.4 drwxr-xr-x 2 lp sys 4096 Nov 27 04:02 cups -rw-r--r-- 1 root root 18903 Nov 11 13:35 dmesg -rw------- 1 root root 29256 Nov 11 08:11 faillog drwxr-xr-x 2 root root 4096 Mar 28 2008 gdm drwx------ 2 root root 4096 Oct 19 04:02 httpd -rw-r--r-- 1 root root 355948 Nov 29 16:34 lastlog drwxr-xr-x 2 root root 4096 Jun 16 15:39 mail -rw------- 1 root root 27520 Nov 29 08:51 maillog -rw------- 1 root root 38980 Nov 23 04:02 maillog.1 -rw------- 1 root root 56964 Nov 16 04:02 maillog.2 -rw------- 1 root root 74842 Nov 9 04:02 maillog.3 -rw------- 1 root root 110136 Nov 2 04:02 maillog.4 -rw------- 1 root root 9165 Nov 29 15:35 messages -rw------- 1 root root 11706 Nov 22 21:30 messages.1 -rw------- 1 root root 35986 Nov 16 03:22 messages.2 -rw------- 1 root root 12430 Nov 8 23:59 messages.3 -rw------- 1 root root 6224 Nov 1 16:21 messages.4 drwxr-xr-x 2 root root 4096 Jun 17 15:02 pm drwx------ 2 root root 4096 Dec 1 2006 ppp drwxr-xr-x 2 root root 4096 Jun 27 2007 prelink -rw-r--r-- 1 root root 31559 Nov 29 04:03 rpmpkgs -rw-r--r-- 1 root root 31559 Nov 22 04:03 rpmpkgs.1 -rw-r--r-- 1 root root 31559 Nov 15 04:03 rpmpkgs.2 -rw-r--r-- 1 root root 31559 Nov 8 04:02 rpmpkgs.3 -rw-r--r-- 1 root root 31559 Nov 1 04:02 rpmpkgs.4 drwx------ 2 root root 4096 May 20 2008 samba -rw-r--r-- 1 root root 107169 Jun 17 15:07 scrollkeeper.log -rw------- 1 root root 1702726 Nov 29 16:34 secure -rw------- 1 root root 5069529 Nov 23 03:38 secure.1 -rw------- 1 root root 1196200 Nov 16 03:30 secure.2 -rw------- 1 root root 2404320 Nov 8 23:59 secure.3 -rw------- 1 root root 6374517 Nov 1 19:52 secure.4 drwxr-xr-x 2 root root 4096 Nov 23 04:02 setroubleshoot -rw------- 1 root root 0 Nov 23 04:02 spooler -rw------- 1 root root 0 Nov 16 04:02 spooler.1 -rw------- 1 root root 0 Nov 9 04:02 spooler.2 -rw------- 1 root root 0 Nov 2 04:02 spooler.3 -rw------- 1 root root 0 Oct 26 04:03 spooler.4 drwxr-x--- 2 squid squid 4096 Apr 1 2008 squid -rw------- 1 root root 0 Jun 17 14:57 tallylog -rw-r--r-- 1 root root 34140 Nov 29 16:34 up2date -rw-r--r-- 1 root root 37324 Nov 23 03:34 up2date.1 -rw-r--r-- 1 root root 43305 Nov 16 03:34 up2date.2 -rw-r--r-- 1 root root 32088 Nov 9 03:49 up2date.3 -rw-r--r-- 1 root root 34650 Nov 2 03:49 up2date.4 drwxr-xr-x 2 root root 4096 Nov 20 2007 vbox -rw-rw-r-- 1 root utmp 23040 Nov 29 16:34 wtmp -rw-rw-r-- 1 root utmp 1093632 Nov 27 02:13 wtmp.1 -rw-rw-r-- 1 root cis90 59894 Oct 24 08:23 Xorg.0.log -rw-rw-r-- 1 root cis90 59894 Sep 16 12:58 Xorg.0.log.old -rw-r--r-- 1 root root 20546 Jun 17 19:32 yum.log [root@opus ~]# How many backups are there of each log? 4 How often are these log files rotated? weekly From observing /var/log …. Log files are owned by root and have restrictive permissions due to the sensitive information they contain 6

7. CIS 191 - Lesson 12 syslog.conf 7

8. CIS 191 - Lesson 12 /etc/syslog.conf /etc/syslog.conf on Opus Each entry is a selector followed by an action 8

9. CIS 191 - Lesson 12 /etc/syslog.conf FacilityDescription authThe authorization system. Ex.: login, su, ftpd, rshd authprivUser access messages use this cronUsed by the cron facility daemonOther daemon programs without a facility of their own ftpUsed by ftp applications kernKernel messages lprThe line printer spooling system mailUsed by mail applications markUsed by syslogd to produce timestamps in log files newsUsed by news applications securitySame as auth. Should not be used anymore. syslog userMessages generated by random user processes. Default. uucpUUCP messages local0 – local7Reserved for local use. *For all 9 http://www.linode.com/wiki/index.php/Syslog_Howto

10. CIS 191 - Lesson 12 /etc/syslog.conf Security Level PriorityKeywordDescription 0emergenciesemerg, panic A panic condition. This is normally broadcast to all users 1alertsalert Inmmediate action required. e.g.: Corrupted system database 2criticalcrit Critical condition. e.g.: Hard device errors 3errorserr, errorError conditions 4warningwarning, warnWarning conditions 5notificationsnotice Normal but significant conditions that need attention 6informationalinfoInformational messages 7debuggingdebugDebugging messages 10 http://www.linode.com/wiki/index.php/Syslog_Howto

11. CIS 191 - Lesson 12 /etc/syslog.conf SelectorDescription kernel.*kernel facility, any priority mail.debug mail facility, debug or higher priority (same as *) lpr,news.*all messages from printer or news auth.warning all security messages of warning or higher priority *.info all messages from any facility of level info or higher ftp.=info ftp facility, info msgs only (and not higher) *.!errany facility, pri <= err only *.!=alertany facility, any priority except alert *.info;mail,news,authpriv.none all msgs with info or higher priority except mail, news, and authpriv mail.2 all mail messages level critical and higher mail.0only critical mail messages 11 http://www.linode.com/wiki/index.php/Syslog_Howto

12. CIS 191 - Lesson 12 /etc/syslog.conf ActionDescription /complete/path/of/some/fileMessages logged to a file /dev/consoleThis is a link to the system console -/complete/path/of/some/file Don't flush (write to disk) file each time; better performance but risks loss of some log info username1[,username2...]Users that will get the message *All logged in users get the message @remotehost Log to remote host. Start the remote syslogd with "-r" option |/path/to/named/pipe To send output to a command you must create a named pipe, say /var/lib/cmd.pipe with the mkfifo command. Then start the command with cmd </var/lib/cmd.pipe 12 http://www.linode.com/wiki/index.php/Syslog_Howto

13. CIS 191 - Lesson 12 Logging Note: You can use the severity level to control where messages are sent, but you don't have control over the level a program assigns to a message. 13

14. CIS 191 - Lesson 12 /etc/syslog.conf write messages to console kernel facility (all messages) commented out 14

15. CIS 191 - Lesson 12 /etc/syslog.conf All facilties with info (6) or higher priority except mail, authpriv or cron write messages to this file 15

16. CIS 191 - Lesson 12 /etc/syslog.conf authpriv facility, any priority write messages to this file 16

17. CIS 191 - Lesson 12 /etc/syslog.conf mail facility, any priority write messages to this file (- means don't flush file each time) 17

18. CIS 191 - Lesson 12 /etc/syslog.conf cron facility, any priority write messages to this file 18

19. CIS 191 - Lesson 12 /etc/syslog.conf All emergency level (0) messages from any facility All logged in users get the message 19

20. CIS 191 - Lesson 12 /etc/syslog.conf Critical (2) or higher messages from uucp or news facilities Messages are written to this file 20

21. CIS 191 - Lesson 12 /etc/syslog.conf any messages from local7 (used by Red Hat family for boot messages) Messages are written to this file 21

22. CIS 191 - Lesson 12 /etc/syslog.conf only notification level (5) messages from any facility Messages are written to this file For Lab 10 In Lab 10 a new entry is added to /etc/syslog.conf for a custom notices log 22

23. CIS 191 - Lesson 12 /etc/syslog.conf Must restart the logging service for the change in /etc/syslog.conf to take effect. For Lab 10 Create a custom logfile 23

24. CIS 191 - Lesson 12 /etc/syslog.conf For Lab 10 Login as root on tty2Login as cis191 on tty3, then su with bad password The new log will hold root logins and login failures 24

25. CIS 191 - Lesson 12 log rotation 25

26. CIS 191 - Lesson 12 Log file rotation logrotate is normally run out of cron once every day 26 The actual program lives in /usr/sbin This is actually a script that calls the logrotate program

27. CIS 191 - Lesson 12 [root@opus ~]# cat /etc/logrotate.conf # see "man logrotate" for details # rotate log files weekly weekly # keep 4 weeks worth of backlogs rotate 4 # create new (empty) log files after rotating old ones create # uncomment this if you want your log files compressed #compress # RPM packages drop log rotation information into this directory include /etc/logrotate.d # no packages own wtmp -- we'll rotate them here /var/log/wtmp { monthly minsize 1M create 0664 root utmp rotate 1 } # system-specific logs may be also be configured here. [root@opus ~]# /etc/logrotate.conf applies to all files for specific files logrotate.conf on Opus 27

28. CIS 191 - Lesson 12 /etc/logrotate.conf [root@benji ~]# cat /etc/logrotate.conf # see "man logrotate" for details # rotate log files weekly weekly # keep 4 weeks worth of backlogs rotate 4 # create new (empty) log files after rotating old ones create # uncomment this if you want your log files compressed #compress # RPM packages drop log rotation information into this directory include /etc/logrotate.d # no packages own wtmp -- we'll rotate them here /var/log/wtmp { monthly create 0664 root utmp rotate 1 } # system-specific logs may be also be configured here. [root@benji ~]# logrotate.conf on Benji 28

29. CIS 191 - Lesson 12 logins 29

30. CIS 191 - Lesson 12 /var/log/wtmp and var/log/btmp [root@opus log]# ls -l /var/log total 153576 -rw-r----- 1 root root 3665 Nov 11 13:36 acpid -rw------- 1 root root 527440 Jun 16 15:47 anaconda.log -rw------- 1 root root 22282 Jun 16 15:47 anaconda.syslog -rw------- 1 root root 58040 Jun 16 15:47 anaconda.xlog drwxr-x--- 2 root root 4096 Nov 24 02:03 audit -rw------- 1 root root 0 Nov 23 04:02 boot.log -rw------- 1 root root 0 Nov 16 04:02 boot.log.1 -rw------- 1 root root 0 Nov 9 04:02 boot.log.2 -rw------- 1 root root 0 Nov 2 04:02 boot.log.3 -rw------- 1 root root 0 Oct 26 04:03 boot.log.4 -rw------- 1 root utmp 136987008 Nov 29 15:16 btmp drwxr-xr-x 2 root root 4096 Jun 28 2007 conman drwxr-xr-x 2 root root 4096 Jun 28 2007 conman.old -rw------- 1 root root 13117 Nov 29 20:01 cron -rw------- 1 root root 13860 Nov 23 04:02 cron.1 -rw------- 1 root root 13706 Nov 16 04:02 cron.2 -rw------- 1 root root 13843 Nov 9 04:02 cron.3 -rw------- 1 root root 14117 Nov 2 04:02 cron.4 drwxr-xr-x 2 lp sys 4096 Nov 27 04:02 cups -rw-r--r-- 1 root root 18903 Nov 11 13:35 dmesg -rw------- 1 root root 29256 Nov 11 08:11 faillog drwxr-xr-x 2 root root 4096 Mar 28 2008 gdm drwx------ 2 root root 4096 Oct 19 04:02 httpd -rw-r--r-- 1 root root 355948 Nov 29 18:39 lastlog drwxr-xr-x 2 root root 4096 Jun 16 15:39 mail -rw------- 1 root root 28085 Nov 29 19:56 maillog -rw------- 1 root root 38980 Nov 23 04:02 maillog.1 -rw------- 1 root root 56964 Nov 16 04:02 maillog.2 -rw------- 1 root root 74842 Nov 9 04:02 maillog.3 -rw------- 1 root root 110136 Nov 2 04:02 maillog.4 -rw------- 1 root root 9165 Nov 29 15:35 messages -rw------- 1 root root 11706 Nov 22 21:30 messages.1 -rw------- 1 root root 35986 Nov 16 03:22 messages.2 -rw------- 1 root root 12430 Nov 8 23:59 messages.3 -rw------- 1 root root 6224 Nov 1 16:21 messages.4 drwxr-xr-x 2 root root 4096 Jun 17 15:02 pm drwx------ 2 root root 4096 Dec 1 2006 ppp drwxr-xr-x 2 root root 4096 Jun 27 2007 prelink -rw-r--r-- 1 root root 31559 Nov 29 04:03 rpmpkgs -rw-r--r-- 1 root root 31559 Nov 22 04:03 rpmpkgs.1 -rw-r--r-- 1 root root 31559 Nov 15 04:03 rpmpkgs.2 -rw-r--r-- 1 root root 31559 Nov 8 04:02 rpmpkgs.3 -rw-r--r-- 1 root root 31559 Nov 1 04:02 rpmpkgs.4 drwx------ 2 root root 4096 May 20 2008 samba -rw-r--r-- 1 root root 107169 Jun 17 15:07 scrollkeeper.log -rw------- 1 root root 1703877 Nov 29 19:59 secure -rw------- 1 root root 5069529 Nov 23 03:38 secure.1 -rw------- 1 root root 1196200 Nov 16 03:30 secure.2 -rw------- 1 root root 2404320 Nov 8 23:59 secure.3 -rw------- 1 root root 6374517 Nov 1 19:52 secure.4 drwxr-xr-x 2 root root 4096 Nov 23 04:02 setroubleshoot -rw------- 1 root root 0 Nov 23 04:02 spooler -rw------- 1 root root 0 Nov 16 04:02 spooler.1 -rw------- 1 root root 0 Nov 9 04:02 spooler.2 -rw------- 1 root root 0 Nov 2 04:02 spooler.3 -rw------- 1 root root 0 Oct 26 04:03 spooler.4 drwxr-x--- 2 squid squid 4096 Apr 1 2008 squid -rw------- 1 root root 0 Jun 17 14:57 tallylog -rw-r--r-- 1 root root 34818 Nov 29 19:34 up2date -rw-r--r-- 1 root root 37324 Nov 23 03:34 up2date.1 -rw-r--r-- 1 root root 43305 Nov 16 03:34 up2date.2 -rw-r--r-- 1 root root 32088 Nov 9 03:49 up2date.3 -rw-r--r-- 1 root root 34650 Nov 2 03:49 up2date.4 drwxr-xr-x 2 root root 4096 Nov 20 2007 vbox -rw-rw-r-- 1 root utmp 26112 Nov 29 19:02 wtmp -rw-rw-r-- 1 root utmp 1093632 Nov 27 02:13 wtmp.1 -rw-rw-r-- 1 root cis90 59894 Oct 24 08:23 Xorg.0.log -rw-rw-r-- 1 root cis90 59894 Sep 16 12:58 Xorg.0.log.old -rw-r--r-- 1 root root 20546 Jun 17 19:32 yum.log [root@opus log]# bad login attempts good login attempts 30

31. CIS 191 - Lesson 12 /var/log/wtmp and var/log/btmp [root@opus ~]# lastb | grep "cool.nju.edu.cn" | head bind ssh:notty cool.nju.edu.cn Sun Nov 30 06:35 - 06:35 (00:00) [root@opus ~]# lastb | grep "cool.nju.edu.cn" | wc -l 3104 [root@opus ~]# 31 Shows break in attempt on 11/30/2008

32. CIS 191 - Lesson 12 /var/log/wtmp and var/log/btmp [root@opus ~]# lastb | grep "Nov 2 17:45" webadmin ssh:notty 211.96.97.179 Sun Nov 2 17:45 - 17:45 (00:00) retsu ssh:notty 211.96.97.179 Sun Nov 2 17:45 - 17:45 (00:00) sbear ssh:notty 211.96.97.179 Sun Nov 2 17:45 - 17:45 (00:00) sky ssh:notty 211.96.97.179 Sun Nov 2 17:45 - 17:45 (00:00) harvey ssh:notty 211.96.97.179 Sun Nov 2 17:45 - 17:45 (00:00) [root@opus ~]# [root@opus ~]# lastb -i | grep "211.96.97.179" | wc -l 598 [root@opus ~]# 32 Shows break in attempt by 211.96.97.179 on 11/2/2008

33. CIS 191 - Lesson 12 /var/log/lastlog 33

34. CIS 191 - Lesson 12 /var/log/lastlog 34

35. CIS 191 - Lesson 12 /var/log/wtmp and var/log/btmp failed logins successful logins 35

36. CIS 191 - Lesson 12 /var/log/wtmp and var/log/btmp Either way prints successful login history 36

37. CIS 191 - Lesson 12 /var/log/wtmp and var/log/btmp [root@opus log]# lastb | sort | cut -f1 -d' ' | grep -v ^$ | uniq –c > bad [root@opus log]# sort –g bad > bad.sort [root@opus log]# [root@opus log]# cat bad.sort | tail -50 471 ftp 472 public 490 test 490 tomcat 498 user 506 service 508 mike 508 username 524 cyrus 530 pgsql 532 test1 544 master 554 linux 554 toor 576 paul 584 support 590 testuser 604 irc 610 test 656 noc 686 www 690 postfix 723 john 734 testing 738 adam 746 alex 754 info 798 tester 832 library 935 guest 990 admin 1002 office 1022 temp 1070 ftpuser 1138 webadmin 1298 nagios 1332 web 1374 a 1384 student 1416 postgres 1690 user 1858 oracle 1944 mysql 2086 webmaste 5324 test 10803 root 10824 admin 18679 root 24064 root [root@opus log]# Top 50 usernames used by the bad guys 37

38. CIS 191 - Lesson 12 /var/log/wtmp and var/log/btmp [root@opus log]# lastb | sort | cut -f1 -d' ' | grep -v ^$| uniq -c | wc -l 22128 [root@opus log]# [root@opus log]# lastb | grep root | wc -l 54117 [root@opus log]# 22128 usernames used and failed 53117 failed root logins Now you know why you need a strong password! 38

39. CIS 191 - Lesson 12 logwatch 39

40. CIS 191 - Lesson 12 logwatch [root@opus ~]# mail Mail version 8.1 6/6/93. Type ? for help. "/var/spool/mail/root": 349 messages 349 new >N 1 logwatch@opus.cabril Mon Jun 16 17:04 43/1587 "Logwatch for opus.cabrillo.edu (Linux)" N 2 root@opus.cabrillo.e Mon Jun 16 17:12 18/795 "Anacron job for 'opus.cabrillo.edu' cron." N 3 logwatch@opus.cabril Tue Jun 17 16:14 141/3966 "Logwatch for opus.cabrillo.edu (Linux)" N 4 logwatch@opus.cabril Wed Jun 18 04:02 728/32707 "Logwatch for opus.cabrillo.edu (Linux)" N 5 root@opus.cabrillo.e Wed Jun 18 04:05 47/1877 "Cron run-parts /etc/cron.dail" N 6 logwatch@opus.cabril Thu Jun 19 04:02 1007/61932 "Logwatch for opus.cabrillo.edu (Linux)" N 7 root@opus.cabrillo.e Thu Jun 19 04:02 47/1889 "Cron run-parts /etc/cron.dail" N 8 logwatch@opus.cabril Fri Jun 20 04:02 168/5533 "Logwatch for opus.cabrillo.edu (Linux)" N 9 root@opus.cabrillo.e Fri Jun 20 04:02 47/1891 "Cron run-parts /etc/cron.dail" N 10 logwatch@opus.cabril Sat Jun 21 04:02 274/8886 "Logwatch for opus.cabrillo.edu (Linux)" N 11 root@opus.cabrillo.e Sat Jun 21 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 12 logwatch@opus.cabril Sun Jun 22 04:02 156/4722 "Logwatch for opus.cabrillo.edu (Linux)" N 13 root@opus.cabrillo.e Sun Jun 22 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 14 logwatch@opus.cabril Mon Jun 23 04:02 241/10770 "Logwatch for opus.cabrillo.edu (Linux)" N 15 root@opus.cabrillo.e Mon Jun 23 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 16 logwatch@opus.cabril Tue Jun 24 04:02 3768/316984 "Logwatch for opus.cabrillo.edu (Linux)" N 17 root@opus.cabrillo.e Tue Jun 24 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 18 logwatch@opus.cabril Wed Jun 25 04:02 3246/274685 "Logwatch for opus.cabrillo.edu (Linux)" N 19 root@opus.cabrillo.e Wed Jun 25 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 20 logwatch@opus.cabril Thu Jun 26 04:02 1390/112446 "Logwatch for opus.cabrillo.edu (Linux)" N 21 root@opus.cabrillo.e Thu Jun 26 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 22 logwatch@opus.cabril Fri Jun 27 04:02 72/2185 "Logwatch for opus.cabrillo.edu (Linux)" N 23 root@opus.cabrillo.e Fri Jun 27 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 24 logwatch@opus.cabril Sat Jun 28 04:02 91/3228 "Logwatch for opus.cabrillo.edu (Linux)" N 25 root@opus.cabrillo.e Sat Jun 28 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 26 logwatch@opus.cabril Sun Jun 29 04:02 150/6673 "Logwatch for opus.cabrillo.edu (Linux)" N 27 root@opus.cabrillo.e Sun Jun 29 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 28 logwatch@opus.cabril Mon Jun 30 04:02 247/14351 "Logwatch for opus.cabrillo.edu (Linux)" N 29 root@opus.cabrillo.e Mon Jun 30 04:02 47/1894 "Cron run-parts /etc/cron.dail" N 30 logwatch@opus.cabril Tue Jul 1 04:02 395/20660 "Logwatch for opus.cabrillo.edu (Linux)" N 31 root@opus.cabrillo.e Tue Jul 1 04:02 47/1891 "Cron run-parts /etc/cron.dail" N 32 logwatch@opus.cabril Wed Jul 2 04:02 481/32664 "Logwatch for opus.cabrillo.edu (Linux)" N 33 root@opus.cabrillo.e Wed Jul 2 04:02 47/1891 "Cron run-parts /etc/cron.dail" N 34 logwatch@opus.cabril Thu Jul 3 04:02 102/3197 "Logwatch for opus.cabrillo.edu (Linux)" N 35 root@opus.cabrillo.e Thu Jul 3 04:02 47/1891 "Cron run-parts /etc/cron.dail" & 29 You have mail … from logwatch 40

41. CIS 191 - Lesson 12 logwatch example email message from logwatch 41 & 11 Message 11: From root@benji.localdomain Tue Dec 2 10:47:06 2008 Date: Tue, 2 Dec 2008 10:47:06 -0800 To: root@benji.localdomain From: logwatch@benji.localdomain Subject: Logwatch for benji.localdomain (Linux) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="iso-8859-1" ################### Logwatch 7.3 (03/24/06) #################### Processing Initiated: Tue Dec 2 10:47:06 2008 Date Range Processed: yesterday ( 2008-Dec-01 ) Period is day. Detail Level of Output: 0 Type of Output: unformatted Logfiles for Host: benji.localdomain ################################################################## --------------------- Disk Space Begin ------------------------ Filesystem Size Used Avail Use% Mounted on /dev/sda1 2.9G 2.5G 272M 91% / /dev/sda5 289M 234M 41M 86% /opt /dev/sda3 487M 77M 385M 17% /var /dev/sda7 196M 5.6M 181M 3% /home ---------------------- Disk Space End ------------------------- ###################### Logwatch End #########################

42. CIS 191 - Lesson 12 Configuring logwatch 42 This file shows all the defaults being used by logwatch Level of detail is Low by default

43. CIS 191 - Lesson 12 Configuring logwatch Edit /etc/logwatch/conf/logwatch.conf to modify defaults 43 This line was added to override the default level of Low Read this for all options to set

44. CIS 191 - Lesson 12 logwatch Message 14: From root@benji.localdomain Tue Dec 2 10:53:22 2008 Date: Tue, 2 Dec 2008 10:53:21 -0800 To: root@benji.localdomain From: logwatch@benji.localdomain Subject: Logwatch for benji.localdomain (Linux) MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="iso-8859-1" ################### Logwatch 7.3 (03/24/06) #################### Processing Initiated: Tue Dec 2 10:53:21 2008 Date Range Processed: yesterday ( 2008-Dec-01 ) Period is day. Detail Level of Output: 10 Type of Output: unformatted Logfiles for Host: benji.localdomain ################################################################## --------------------- Cron Begin ------------------------ Commands Run: User root: /sbin/dump 0uf /backup/level0/backup-L0-`date +%Y-%d-%m`.dmp /home: 2 Time(s) /sbin/dump 1uf /backup/level1/backup-L1-`date +%Y-%d-%m`.dmp /home: 5 Time(s) /sbin/dump 2uf /backup/level2/backup-L2-`date '+: 4 Time(s) /sbin/dump 2uf /backup/level2/backup-L2-`date +: 5 Time(s) /sbin/dump 2uf /backup/level2/backup-L2-`date +%Y-%d-%m`.dmp /home: 14 Time(s) /sbin/dump 2uf /backup/level2/backup-L2.dmp /home: 2 Time(s) dump 1uf /backup/level1/backup-daily-$(date +: 1 Time(s) dump 2uf /backup/level2/backup-L2.dmp /home: 2 Time(s) dump 2uf /backup/level2/backup-daily-$(date +: 9 Time(s) logwatch report using High level of detail 44

45. CIS 191 - Lesson 12 logwatch personal crontab deleted: 3 Time(s) personal crontab edited: 6 Time(s) personal crontab listed: 7 Time(s) personal crontab reloaded: 7 Time(s) personal crontab replaced: 11 Time(s) run-parts /etc/cron.daily: 1 Time(s) run-parts /etc/cron.hourly: 24 Time(s) run-parts /etc/cron.monthly: 1 Time(s) ---------------------- Cron End ------------------------- --------------------- sendmail Begin ------------------------ STATISTICS ---------- Bytes Transferred: 90737 Messages Processed: 92 Addressed Recipients: 92 Message recipients per delivery agent: Name # Rcpts local 46 --------------------- TOTAL: 46 in addition to 46 relay submission(s) from MSP 45 logwatch report using High level of detail continued

46. CIS 191 - Lesson 12 logwatch Message Size Distribution: Range # Msgs KBytes 0 - 10k 92 88 ---------------------------------- TOTAL 92 88 Avg. Size 0 Top 10 Email Recipients ---------------------------------- root@benji.localdomain : 46 emails Top relays (recipients/connections - min 10 rcpts, max 25 lines): 46/46: benji.localdomain [127.0.0.1] 46/46: root@localhost ---------------------- sendmail End ------------------------- --------------------- Syslogd Begin ------------------------ Syslogd started 1 Time(s) ---------------------- Syslogd End ------------------------- 46 logwatch report using High level of detail continued

47. CIS 191 - Lesson 12 logwatch --------------------- Disk Space Begin ------------------------ Filesystem Size Used Avail Use% Mounted on /dev/sda1 2.9G 2.5G 272M 91% / /dev/sda5 289M 234M 41M 86% /opt /dev/sda3 487M 77M 385M 17% /var /dev/sda7 196M 5.6M 181M 3% /home ---------------------- Disk Space End ------------------------- ###################### Logwatch End ######################### & 47 logwatch report using High level of detail continued

48. CIS 191 - Lesson 12 logwatch --------------------- SSHD Begin ------------------------ SSHD Killed: 1 Time(s) SSHD Started: 1 Time(s) Disconnecting after too many authentication failures for user: guest90 : 1 Time(s) Failed logins from: 76.254.22.196 (adsl-76-254-22-196.dsl.pltn13.sbcglobal.net): 2 times 201.7.115.194 (201-7-115-194.spopa302.ipd.brasiltelecom.net.br): 2135 times 210.240.12.14: 20 times Illegal users from: 201.7.115.194 (201-7-115-194.spopa302.ipd.brasiltelecom.net.br): 564 times 210.240.12.14: 42 times Users logging in through sshd: guest: 76.254.22.196 (adsl-76-254-22-196.dsl.pltn13.sbcglobal.net): 2 times jimg: 70.132.20.25 (adsl-70-132-20-25.dsl.snfc21.sbcglobal.net): 7 times ordazedw: 76.254.22.196 (adsl-76-254-22-196.dsl.pltn13.sbcglobal.net): 1 time root: 63.249.86.11 (dsl-63-249-86-11.cruzio.com): 3 times 70.132.20.25 (adsl-70-132-20-25.dsl.snfc21.sbcglobal.net): 1 time rsimms: 63.249.86.11 (dsl-63-249-86-11.cruzio.com): 2 times the bad boys trying to break in … this is why you need strong passwords 48

49. CIS 191 - Lesson 12 logwatch 49 http://ws.arin.net/whois/?queryinput=201.7.115.194

50. CIS 191 - Lesson 12 /var/log/secure The bad boys trying to break in as root … this is why you need strong passwords Nov 30 06:02:24 opus sshd[27486]: Failed password for root from 202.119.60.132 port 36322 ssh2 Nov 30 06:02:24 opus sshd[27487]: Received disconnect from 202.119.60.132: 11: Bye Bye Nov 30 06:02:27 opus sshd[27488]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn user=root Nov 30 06:02:29 opus sshd[27488]: Failed password for root from 202.119.60.132 port 36846 ssh2 Nov 30 06:02:29 opus sshd[27489]: Received disconnect from 202.119.60.132: 11: Bye Bye Nov 30 06:02:32 opus sshd[27490]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn user=root Nov 30 06:02:33 opus sshd[27490]: Failed password for root from 202.119.60.132 port 37480 ssh2 Nov 30 06:02:34 opus sshd[27491]: Received disconnect from 202.119.60.132: 11: Bye Bye Nov 30 06:02:36 opus sshd[27492]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn user=root Nov 30 06:02:38 opus sshd[27492]: Failed password for root from 202.119.60.132 port 38030 ssh2 Nov 30 06:02:39 opus sshd[27493]: Received disconnect from 202.119.60.132: 11: Bye Bye Nov 30 06:02:42 opus sshd[27494]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn user=root Nov 30 06:02:43 opus sshd[27494]: Failed password for root from 202.119.60.132 port 38679 ssh2 Nov 30 06:02:43 opus sshd[27495]: Received disconnect from 202.119.60.132: 11: Bye Bye Nov 30 06:02:46 opus sshd[27496]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn user=root Nov 30 06:02:48 opus sshd[27496]: Failed password for root from 202.119.60.132 port 39448 ssh2 Nov 30 06:02:48 opus sshd[27497]: Received disconnect from 202.119.60.132: 11: Bye Bye Nov 30 06:02:50 opus sshd[27498]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn user=root 50

51. CIS 191 - Lesson 12 /var/log/secure The bad boys trying to break in, guessing usernames … this is why you need strong passwords Nov 30 06:27:20 opus sshd[28166]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn Nov 30 06:27:20 opus sshd[28166]: pam_succeed_if(sshd:auth): error retrieving information about user shop Nov 30 06:27:23 opus sshd[28166]: Failed password for invalid user shop from 202.119.60.132 port 40634 ssh2 Nov 30 06:27:23 opus sshd[28167]: Received disconnect from 202.119.60.132: 11: Bye Bye Nov 30 06:27:25 opus sshd[28168]: Invalid user lady from 202.119.60.132 Nov 30 06:27:25 opus sshd[28169]: input_userauth_request: invalid user lady Nov 30 06:27:25 opus sshd[28168]: pam_unix(sshd:auth): check pass; user unknown Nov 30 06:27:25 opus sshd[28168]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn Nov 30 06:27:25 opus sshd[28168]: pam_succeed_if(sshd:auth): error retrieving information about user lady Nov 30 06:27:28 opus sshd[28168]: Failed password for invalid user lady from 202.119.60.132 port 41408 ssh2 Nov 30 06:27:28 opus sshd[28169]: Received disconnect from 202.119.60.132: 11: Bye Bye Nov 30 06:27:30 opus sshd[28170]: Invalid user lady from 202.119.60.132 Nov 30 06:27:30 opus sshd[28171]: input_userauth_request: invalid user lady Nov 30 06:27:30 opus sshd[28170]: pam_unix(sshd:auth): check pass; user unknown Nov 30 06:27:30 opus sshd[28170]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=cool.nju.edu.cn Nov 30 06:27:30 opus sshd[28170]: pam_succeed_if(sshd:auth): error retrieving information about user lady 51