1. Copyright 2006 Richard Bejtlich and David Bianco 1 Network Security Monitoring with Sguil Richard Bejtlich richard@taosecurity.com www.taosecurity.com / taosecurity.blogspot.com David Bianco david@vorant.com www.vorant.com / infosecpotpourri.blogspot.com ShmooCon 2006

2. Copyright 2006 Richard Bejtlich and David Bianco 2 Overview The Problem Network Security Monitoring Sguil Case Study

3. Copyright 2006 Richard Bejtlich and David Bianco 3 The Problem You want to know if your network is compromised You could... inspect every host for signs of compromise –Where to begin? What to check? Do you trust results? –Unless your enterprise is very small, and you are competent to perform host-centric forensics, this is not a viable option You could... perform a vulnerability assessment –VA indirectly measures compromise by showing potential for intrusion, not actual intrusions –VA is vulnerability-centric, not threat-centric; not recommended You could... inspect network traffic for signs of compromise –Where to monitor? What about encryption or high bandwidth? –This option represents best return on invested resources, if the right data is collected, analyzed, and escalated

4. Copyright 2006 Richard Bejtlich and David Bianco 4 The Problem Most people install an IDS or IPS in monitoring mode and wait for alerts Thanks to the IDS alert, an analyst is aware of a Web site problem -- but what about activity before or after the alert? EventExampleIDS Action Event 1Ping Web site IP addressIgnore Event 2Visit Web siteIgnore Event 3Exploit Web site flawAlert Event 4FTP to retrieve toolsIgnore Event 5Install back doorIgnore Event 6Communicate with back doorIgnore Event 7Connect via SSH to another siteIgnore Event 8Transfer local exploit via SCPIgnore Event...And so on...?

5. Copyright 2006 Richard Bejtlich and David Bianco 5 The Problem The previous slide presented a best-case scenario -- at least the attack was detected by the IDS! But what do you get with that alert? –Cryptic message about an attack –Maybe a packet that specifically triggered an alert –A reference to visit the vendor's Web site for more generic info Factors compounding the problem –Attack over HTTPS using SSL –Attack using insertion and evasion methods –Attack using a zero-day exploit undetected by any IDS Scarier scenarios –Use stolen credentials and connect via SSH –Compromise a customer or employee and ride their VPN –Go rogue and steal from your own company

6. Copyright 2006 Richard Bejtlich and David Bianco 6 The Problem Many security developers and vendors believe one or more of the following –Attacks can be understood prior to execution –Methods to detect or prevent attacks can be encapsulated in programming logic –Customers will purchase, properly configure, and effectively deploy products offering sufficient defensive logic –The customer's environment will behave as anticipated by the developers and vendors Accordingly, developers and vendors field alert-centric products which act on those beliefs All of these beliefs must hold true in order to counter sophisticated threats, but few do

7. Copyright 2006 Richard Bejtlich and David Bianco 7 The Problem Investigations with alert-centric systems quickly end, often without resolving the incident Analysts stuck with only alert data to inspect cannot make validation and escalation decisions –MSSPs call customers to ask if they have been compromised –Security personnel ignore alerts because they have no other data ALERT Queries database for alerts Analyst sees original alert Database returns single alert Investigation ends

8. Copyright 2006 Richard Bejtlich and David Bianco 8 Network Security Monitoring Network security monitoring is the collection, analysis and escalation of indications and warning to detect and respond to intrusions NSM gives analysts the data they need to make decisions NSM treats all data as indicators, not "false positives" or "false negatives"

9. Copyright 2006 Richard Bejtlich and David Bianco 9 Network Security Monitoring NSM relies upon four forms of traffic-centric data –Statistical data (Capinfos, Tcpdstat) Descriptive, high-level view of aggregated events –Session data (Argus, SANCP, NetFlow) Summaries of conversations between systems Content-neutral, compact; encryption no problem –Full content data (Tcpdump, Tethereal, Snort as packet logger) All packet details, including application layer Expensive to save, but always most granular analysis –Alert data (Snort, Bro, other IDSs) Traditional IDS alerts or judgments (“RPC call!”) Context-sensitive, either by signature or anomaly Sguil (www.sguil.net) is an interface to much of this in a single open source suite

10. Copyright 2006 Richard Bejtlich and David Bianco 10 Network Security Monitoring Alert data Session data Full content dataStatistical data

11. Copyright 2006 Richard Bejtlich and David Bianco 11 Network Security Monitoring Revisit intrusion scenario when NSM data is available Analysts have much more data to review * if unencrypted (more common than you might think) EventExampleIDS ActionHelpful NSM Collection Event 1Ping Web site IP addressIgnoreSession Event 2Visit Web siteIgnoreSession, Full Content Event 3Exploit Web site flawAlertAlert, Session Event 4FTP to retrieve toolsIgnoreSession, Full Content Event 5Install back doorIgnoreSession, Full Content Event 6Communicate with back doorIgnoreSession, Full Content * Event 7Connect via SSH to another siteIgnoreSession Event 8Transfer local exploit via SCPIgnoreSession Event...And so on...?

12. Copyright 2006 Richard Bejtlich and David Bianco 12 Network Security Monitoring Investigations with NSM present many more options ALERT Queries database for alerts Analyst sees original alert Database returns single alert Queries database for sessions SESSIONS Analyst sees FTP to retrieve tools Reconstructs FTP control and data channels FULL CONTENT Queries database for sessions FTP data channel allows analysis of intruder back door SESSIONS Analyst sees connections to other IPs and so on...

13. Copyright 2006 Richard Bejtlich and David Bianco 13 Network Security Monitoring NSM does not try to anticipate attacks NSM uses a "dumb is smart" approach –NSM does not rely on fancy systems to pass judgements on network traffic, to the exclusion of all other collection mechanisms –NSM does leverage smart systems (IDS, network anomaly detection, etc.) for initial clues NSM session and full content collection is completely content neutral –Session and full content data are collected whether or not any other system thinks they are interesting NSM is not SIM/SEM: a SIM/SEM collects and correlates log sources which may or may not have any value

14. Copyright 2006 Richard Bejtlich and David Bianco 14 Sguil Sguil is an open source interface to NSM data Lead developer: Bamm Visscher Ancestry: Snort Personal REal-time GUI (SPREG), circa 2001 Sguil released as open source at sguil.sf.net in 2003 Version 0.6.0p1 released 1 Dec 2005 Coded mainly in Tcl/Tk Integrates: –Alert data from Snort –Session data from SANCP –Full content data from a second instance of Snort

15. Copyright 2006 Richard Bejtlich and David Bianco 15 Sguil Sensor watches traffic Sguil client is dynamic Tcl/Tk GUI on Windows or UNIX Network traffic on monitored link MySQL database stores alert and session data Log_packets.sh collects full content data, provided on demand via sensor_agent.tcl SANCP (www.metre.net/sancp.html) collects session data, sent to database via sensor_agent.tcl Snort sends alert data output to Barnyard (www.sf.net/projects/barnyard) Barnyard sends output to sensor_agent.tcl Sensor_agent.tcl coordinates data flow with sguild On Sguil server, sguild answers requests from Sguil client

16. Copyright 2006 Richard Bejtlich and David Bianco 16 Sguil Launch sguil.tk, and enter Sguild host, port, username, and password Select a sensor, then Start Sguil

17. Copyright 2006 Richard Bejtlich and David Bianco 17 Sguil

18. Copyright 2006 Richard Bejtlich and David Bianco 18

19. Copyright 2006 Richard Bejtlich and David Bianco 19

20. Copyright 2006 Richard Bejtlich and David Bianco 20

21. Copyright 2006 Richard Bejtlich and David Bianco 21 Sguil FTP Data Channel shows source code FTP control channel shows commands

22. Copyright 2006 Richard Bejtlich and David Bianco 22 How to Pwn a Million PCs Without Breaking a Sweat A Sguil Case Study

23. Copyright 2006 Richard Bejtlich and David Bianco 23 Sguil/NSM Case Study Study based on an exploit encountered “in the wild” The exploit used the WMF vulnerability Delivered via a popunder ad while victim was visiting an otherwise legit website This case study recreates my incident research process to show off the power of sguil High-level writeup available on my blog: –http://infosecpotpourri.blogspot.com/2006/01/how-to-pwn-million- computers-without.html –Aimed towards users/managers Saved the good stuff for ShmooCon!

24. Copyright 2006 Richard Bejtlich and David Bianco 24 Important Notes The victim’s identity has been obfuscated to protect the innocent The ad servers’ identities have been obfuscated to protect the guilty and the not-so-guilty Some URLs have been obfuscated to protect the silly Legitimate website names appearing in this presentation have nothing to do with this exploit and are only there to provide context for understanding the web session

25. Copyright 2006 Richard Bejtlich and David Bianco 25 “It was a dark and stormy night…”

26. Copyright 2006 Richard Bejtlich and David Bianco 26 Was that a real exploit I just saw?

27. Copyright 2006 Richard Bejtlich and David Bianco 27 Was that a real exploit I just saw?

28. Copyright 2006 Richard Bejtlich and David Bianco 28 What other events were generated?

29. Copyright 2006 Richard Bejtlich and David Bianco 29 Quick session check (source)

30. Copyright 2006 Richard Bejtlich and David Bianco 30 Quick session check (victim)

31. Copyright 2006 Richard Bejtlich and David Bianco 31 “I will hunt you down…” Also cross-checked other sources, such as: –Antivirus logs –Manual AV update and scan –Checked system for c:\n.exe as specified in WMF file Exploit attempt seems to have been unsuccessful Crisis averted, but let’s have some fun! All the sessions are HTTP, so we can leverage that to help us reconstruct the sequence of events Begin with the transcript of the exploit session Match up “Referrer” tags with requests and work backwards –Like climbing a ladder

32. Copyright 2006 Richard Bejtlich and David Bianco 32 Victim’s Session List

33. Copyright 2006 Richard Bejtlich and David Bianco 33 Rung #1: Exploit Delivered

34. Copyright 2006 Richard Bejtlich and David Bianco 34 Rung #2: Spf99 Serves the Ad

35. Copyright 2006 Richard Bejtlich and David Bianco 35 Victim’s Session List

36. Copyright 2006 Richard Bejtlich and David Bianco 36 Rung #3: Cash4popupads Handoff to Spf99

37. Copyright 2006 Richard Bejtlich and David Bianco 37 Victim’s Session List

38. Copyright 2006 Richard Bejtlich and David Bianco 38 Rung #4: Cash4popupads creates a popunder

39. Copyright 2006 Richard Bejtlich and David Bianco 39 Victim’s Session List

40. Copyright 2006 Richard Bejtlich and David Bianco 40 Rung #5: A Legit Site (HTMHelper)

41. Copyright 2006 Richard Bejtlich and David Bianco 41 HTMHelper Page Source <SCRIPT LANGUAGE="JavaScript1.1" SRC="http://popunder.Cash4popupads.com/popup.php?id=XXXX">

42. Copyright 2006 Richard Bejtlich and David Bianco 42 Victim’s Session List

43. Copyright 2006 Richard Bejtlich and David Bianco 43 Rung #6: Another Legit Site (MySpace)

44. Copyright 2006 Richard Bejtlich and David Bianco 44 MySpace Page Source <div style="position:absolute; left:0px; top:0px; width:88px; height:31px;"> <img src="http://www.htmhelper.com/onlineicon/clickhere.gif" alt="Myspace Online Icon" />

45. Copyright 2006 Richard Bejtlich and David Bianco 45 “Insert Tab A into Slot B…” Victim browses a MySpace profile page –The page owner or one of the commenters is online, and has the “online status” icon showing by their name. –The status icon is provided by and linked back to the HTMHelper site The HTMHelper page is ad-supported and contains a JavaScript snippet to display popunder ads from Cash4popupads. This may be annoying, but not intrinsically malicious Cash4popupads establishes the popunder window but not the ad content –It’s acting more as a conduit for the ads, which are provided by Spf99

46. Copyright 2006 Richard Bejtlich and David Bianco 46 “Score along line C and fold to meet side D…” Spf99 served the actual infected file –101.wmf Internal codes indicate this was provided by “affiliate 101” –Could be an individual –Could be another ad network –Who knows? This is the top of the ladder (for now) How would you continue the investigation?

47. Copyright 2006 Richard Bejtlich and David Bianco 47 Sguil Simplest way to try Sguil: use Win client, demo server –Install ActiveState TCL (www.activestate.com/Products/Download/Download.plex?id=ActiveTcl) –Visit www.sguil.net, download, extract sguil-client-0.6.0p1.zip –Create a c:\tmp directory –Edit sguil.tk set VERSION "SGUIL-0.6.0" –Edit sguil.conf to match Windows environment # win32 example set ETHEREAL_PATH "c:/progra~1/ethereal/ethereal.exe" # Where to save the temporary raw data files on the client system # You need to remember to delete these yourself. # set ETHEREAL_STORE_DIR /tmp # win32 example set ETHEREAL_STORE_DIR "c:/tmp" # Favorite browser for looking at sig info on snort.org # set BROWSER_PATH /usr/bin/mozilla # win32 example (IE) set BROWSER_PATH c:/progra~1/intern~1/iexplore.exe –Launch Sguil client, connect to demo.sguil.net on port 7734 with any username and password

48. Copyright 2006 Richard Bejtlich and David Bianco 48 Sguil Simplest way to try Sguil with a local setup: use Sguil server VM or client and server VM –Described here: sguil.sourceforge.net/index.php?page=vm –Use Sguil client as described on previous slide to connect to VM –Use complete Sguil VM Other options –InstantNSM (instantnsm.sf.net) –FreeBSD installation script (taosecurity.blogspot.com/2006/01/sguil-installation-script- v0.html) Help available on irc.freenode.net #snort-gui channel Free Tao of Network Security Monitoring chapters on NSM & Sguil in.pdf at www.taosecurity.com/books.html

49. Copyright 2006 Richard Bejtlich and David Bianco 49 Questions? Richard Bejtlich richard@taosecurity.com www.taosecurity.com / taosecurity.blogspot.com David Bianco david@vorant.com www.vorant.com / infosecpotpourri.blogspot.com www.sguil.net irc.freenode.net / #snort-gui