1. Sniffing network traffic in Python
Jose Nazario, Ph.D.

2. Why Python? Interpreted language Rapid development
Bound to be slower than C
Rapid development
Easy data structure use
Fewer LoC per tool
Easy to manipulate strings

3. Marrying Python and Sniffing
Librares in C
Often SWIGged, exported to Python
pcap, dnet, nids …
Modules
pypcap/pcappy – pcap for python
dpkt – packet deconstruction library
libdnet – packet construction library (has python bindings in the distribution)
pynids – connection reassembly tool

4. libnids – reassemble IP streams
NIDS “E” box (event generation box)
Userland TCP/IP stack
Based on Linux IP stack
Uses libpcap, libnet internally
IP fragment reassembly

5. Userland
Kernel
IP stack

6. Userland
Kernel
IP stack
Libnids
IP stack

7. libnids Basics Initialize Register callbacks Run! React nids_init()
nids_register_tcp()
nids_regster_ip()
nids_regiser_udp()
Run!
nids_run()
React
nids_kill_tcp()

8. nids_run() TCP callback UDP callback IP callback TCP stream object:
- TCP state
- client data
- server data
- source IP, port
- dest IP, port
- seq, ack, etc …
UDP packet:
- source IP, port
- dest IP, port
- UDP payload
IP packet
- struct IP packet
- contains upper
layers

9. libnids TCP states NIDS_JUST_ESTABLISHED NIDS_DATA
New TCP connected state (3WHS)
Must set stream->{client,server}.collect=1 to get stream payload collected
NIDS_DATA
Data within a known, established TCP connection
NIDS_RESET, NIDS_CLOSE, NIDS_TIMED_OUT
TCP connection is reset, closed gracefully, or was lost
libnids doesn’t expose SYN_SENT, FIN_WAIT, etc …

10. pynids Basics Event driven interface (nids_run(), nids_next())
TCP stream reassembly
TCP state exposure
Creates a TCP object
Holds addresses, data, etc
UDP and IP packet reassembly

11. Basic pynids Steps Initialize Establish parameters Register callbacks
nids_init()
Establish parameters
nids.param(“attribute”, value)
Register callbacks
nids.register_tcp(handleTcp)
def handleTcp(tcp): …
Go!
nids_run()
while 1: nids_next()

12. pynids Order of Operations
Packets come in
TCP?
State exist? Create state or reuse state
Append data
Process based on state in callback
UDP or IP?
Use handler, pass packet in
You process in callback

13. Code Example (Python) import nids <handleTcpStream> def main():
nids.param("scan_num_hosts", 0)
if not nids.init():
print "error -", nids.errbuf()
sys.exit(1)
nids.register_tcp(handleTcpStream)
try: nids.run() # loop forever
except KeyboardInterrupt:

14. Code Example (Python) cont
def handleTcpStream(tcp):
if tcp.nids_state == nids.NIDS_JUST_EST:
if dport in (80, 8000, 8080):
tcp.client.collect = 1
tcp.server.collect = 1
elif tcp.nids_state == nids.NIDS_DATA:
tcp.discard(0)
elif tcp.nids_state in end_states:
print "addr:", tcp.addr
# may be binary
print "To server:“, tcp.server.data
print "To client:“, tcp.client.data

15. Code Example (C) int main(int argv, char *argv[]) {
if (nids_init() == 0)
err(1, “error, %s”, nids_errbuf);
nids_register_tcp(handleTcp);
nids_run();
exit(0); }

16. Code Example (C), cont
int handleTcp(struct tcp_stream *tcp) {
switch (tcp->nids_state) {
case ‘NIDS_JUST_EST’:
if ((tcp->addr.dest == 80) ||
(tcp->addr.dest == 8000) ||
(tcp->addr.dest == 8080) {
tcp.server.collect = 1;
tcp.client.collect = 1; }
break;
case ‘NIDS_DATA’:
nids_discard(tcp, 0);
case ‘NIDS_CLOSE’:
case ‘NIDS_RESET’:
case ‘NIDS_TIMED_OUT’:
printf(“((%s, %d), (%s, %d))\n”, inet_ntoa(tcp->saddr), tcp.srce,
inet_ntoa(tcp->daddr), tcp.dest);
printf(“%s\n”, tcp->server.data);
printf(“%s\n”, tcp->client.data);
About the same LoC, until we start string manipulation

17. VersionDetect Small python tool Reports on headers Fully passive
Support for: SSH (client, server), WWW (client, server), and SMTP clients
Motivation: coordinate data collection with TCP stack fingerprinting
SymbianOS 6048 (on Nokia 7650?) www /tcp
: 80: Microsoft-IIS/6.0

18. VersionDetect Output 192.168.1.7: 22: SSH-2.0-OpenSSH_3.5
:http: Mozilla/5.0 (X11; U; OpenBSD i386; en-
US; rv:1.5a) Gecko/ Mozilla Firebird/0.6.1
: 80: Microsoft-IIS/5.0
: 80: Netscape-Enterprise/3.6 SP2
: 80: Microsoft-IIS/5.0
: 80: Microsoft-IIS/5.0
: 80: ZEDO 3G
: 80: Apache/ (Unix) PHP/4.3.3
: 80: Apache/ (Unix)
: 80: Apache/ (Unix)
: 80: Apache/ (Unix)
: 80: thttpd/2.25b 29dec2003
: 80: Apache/ (Unix)
mod_perl/1.27
: 80: CAFE/1.0

19. http-graph Small, passive python tool Examines HTTP request header:
GET /blog/styles-site.css HTTP/1.1
Host:
User-Agent: Mozilla/5.0 (X11; U; OpenBSD i386; en-US; rv:1.5a) Gecko/ Mozilla Firebird/0.6.1Accept: text/css,*/*;q=0.1
Referer:

20. http-graph Directed graph history of browsing
Reconstructs graph from referrer and URL in the header:
Referrer Request
Lets you view your history as you took it
Shows natural “hubs” of information
See also:

21. Displaying http-graph Output
Writes a small “dot” file
“dot” part of “graphviz” tool
Use “neato” to graph
Output formats: SVG, PS, PDF, image map
Can make fully interactive!

22. Example http-graph Output

23. Grabbing Data with pynids
tcp.{server, client}.data and just strings
Any string operations will work
Searching
if “HTTP/1.0” in tcp.client.data:
Regular Expression searches
if re.search(“HTTP/1.[10]”, tcp.client.data):
Rewriting
string.replace(req, “GET HTTP/1.0”, “”, 1)

24. More Fun! Privacy invasion Log conversations Steal files
Snarf mail
Log conversations
IRC, AIM, etc …
Steal files
FTP, P2P apps, HTTP downloads …
Disrupt sessions
tcp.kill()
New dsniff is written in Python …

25. flowgrep Marries sniffing with regular expressions
A lot like ngrep, tcpkill, and dsniff
Logs the whole connection, not just a packet
Look for data in streams using regular expressions
Log or kill selected streams
Dirt cheap IDS or IPS
Under 400 lines of code

26. Resources http://www.tcpdump.org/
pycap}

27. Additional Resources Stevens, TCP/IP Illustrated vols 1 and 2
Schiffman, Building Open Source Network Security Tools
RFCs from the IETF