sniffing network traffic in python jose nazario, ph.d
TRANSCRIPT
![Page 2: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/2.jpg)
Why Python?
• Interpreted language– Bound to be slower than C
• Rapid development• Easy data structure use• Fewer LoC per tool• Easy to manipulate strings• http://www.python.org/
![Page 3: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/3.jpg)
Marrying Python and Sniffing
• Librares in C– Often SWIGged, exported to Python– pcap, dnet, nids …
• Modules – pypcap/pcappy – pcap for python– dpkt – packet deconstruction library– libdnet – packet construction library (has python
bindings in the distribution)
– pynids – connection reassembly tool
![Page 4: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/4.jpg)
libnids – reassemble IP streams
NIDS “E” box (event generation box)Userland TCP/IP stackBased on Linux 2.0.36 IP stackUses libpcap, libnet internallyIP fragment reassembly
![Page 5: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/5.jpg)
Kernel
IP stack
Userland
![Page 6: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/6.jpg)
Kernel
IP stack
Userland
IP stackLibnids
![Page 7: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/7.jpg)
libnids Basics
• Initialize– nids_init()
• Register callbacks– nids_register_tcp()– nids_regster_ip()– nids_regiser_udp()
• Run!– nids_run()
• React– nids_kill_tcp()
![Page 8: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/8.jpg)
nids_run()
UDP callbackTCP callback IP callback
TCP stream object: - TCP state - client data - server data - source IP, port - dest IP, port - seq, ack, etc …
UDP packet: - source IP, port - dest IP, port - UDP payload
IP packet - struct IP packet - contains upper layers
![Page 9: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/9.jpg)
libnids TCP states• NIDS_JUST_ESTABLISHED
– New TCP connected state (3WHS)– Must set stream->{client,server}.collect=1
to get stream payload collected
• NIDS_DATA– Data within a known, established TCP connection
• NIDS_RESET, NIDS_CLOSE, NIDS_TIMED_OUT– TCP connection is reset, closed gracefully, or was lost
libnids doesn’t expose SYN_SENT, FIN_WAIT, etc …
![Page 10: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/10.jpg)
pynids Basics
• Event driven interface (nids_run(), nids_next())– TCP stream reassembly– TCP state exposure– Creates a TCP object
• Holds addresses, data, etc – UDP and IP packet reassembly
![Page 11: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/11.jpg)
Basic pynids Steps
• Initialize– nids_init()
• Establish parameters– nids.param(“attribute”, value)
• Register callbacks– nids.register_tcp(handleTcp)– def handleTcp(tcp): …
• Go!– nids_run()– while 1: nids_next()
![Page 12: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/12.jpg)
pynids Order of Operations
• Packets come in
• TCP?– State exist? Create state or reuse state– Append data– Process based on state in callback
• UDP or IP?– Use handler, pass packet in– You process in callback
![Page 13: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/13.jpg)
Code Example (Python)
import nids<handleTcpStream>
def main(): nids.param("scan_num_hosts", 0) if not nids.init(): print "error -", nids.errbuf() sys.exit(1) nids.register_tcp(handleTcpStream) try: nids.run() # loop forever except KeyboardInterrupt: sys.exit(1)
![Page 14: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/14.jpg)
Code Example (Python) cont
def handleTcpStream(tcp): if tcp.nids_state == nids.NIDS_JUST_EST: if dport in (80, 8000, 8080): tcp.client.collect = 1 tcp.server.collect = 1 elif tcp.nids_state == nids.NIDS_DATA: tcp.discard(0) elif tcp.nids_state in end_states: print "addr:", tcp.addr # may be binary print "To server:“, tcp.server.data print "To client:“, tcp.client.data
![Page 15: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/15.jpg)
Code Example (C)
int main(int argv, char *argv[])
{
if (nids_init() == 0)
err(1, “error, %s”, nids_errbuf);
nids_register_tcp(handleTcp);
nids_run();
exit(0);
}
![Page 16: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/16.jpg)
Code Example (C), contint handleTcp(struct tcp_stream *tcp) { switch (tcp->nids_state) { case ‘NIDS_JUST_EST’: if ((tcp->addr.dest == 80) || (tcp->addr.dest == 8000) || (tcp->addr.dest == 8080) { tcp.server.collect = 1; tcp.client.collect = 1; } break; case ‘NIDS_DATA’: nids_discard(tcp, 0); break; case ‘NIDS_CLOSE’: case ‘NIDS_RESET’: case ‘NIDS_TIMED_OUT’: printf(“((%s, %d), (%s, %d))\n”, inet_ntoa(tcp->saddr), tcp.srce,
inet_ntoa(tcp->daddr), tcp.dest); printf(“%s\n”, tcp->server.data); printf(“%s\n”, tcp->client.data); break; }}
About the same LoC, until we start string manipulation
![Page 17: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/17.jpg)
VersionDetect
• Small python tool
• Reports on headers
• Fully passive– Support for: SSH (client, server), WWW
(client, server), and SMTP clients
• Motivation: coordinate data collection with TCP stack fingerprinting
63.236.16.161 SymbianOS 6048 (on Nokia 7650?) www 80/tcp 63.236.16.161: 80: Microsoft-IIS/6.0
![Page 18: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/18.jpg)
VersionDetect Output 192.168.1.7: 22: SSH-2.0-OpenSSH_3.5 192.168.1.101:http: Mozilla/5.0 (X11; U; OpenBSD i386; en-
US; rv:1.5a) Gecko/20031030 Mozilla Firebird/0.6.1 168.75.65.85: 80: Microsoft-IIS/5.0 165.1.76.60: 80: Netscape-Enterprise/3.6 SP2 168.75.65.69: 80: Microsoft-IIS/5.0 168.75.65.87: 80: Microsoft-IIS/5.0 69.28.159.7: 80: ZEDO 3G 198.65.148.234: 80: Apache/1.3.29 (Unix) PHP/4.3.3 216.150.209.231: 80: Apache/1.3.31 (Unix) 212.187.153.30: 80: Apache/1.3.31 (Unix) 212.187.153.37: 80: Apache/1.3.31 (Unix) 212.187.153.32: 80: thttpd/2.25b 29dec2003 64.209.232.207: 80: Apache/1.3.27 (Unix) mod_perl/1.27 216.239.39.99: 80: CAFE/1.0
![Page 19: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/19.jpg)
http-graph
• Small, passive python tool
• Examines HTTP request header:
GET /blog/styles-site.css HTTP/1.1
Host: www.jackcheng.com
User-Agent: Mozilla/5.0 (X11; U; OpenBSD i386; en-US; rv:1.5a) Gecko/20031030 Mozilla Firebird/0.6.1Accept: text/css,*/*;q=0.1
Referer: http://www.jackcheng.com/blog/archives/2004/12/ipod_rumors.html
![Page 20: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/20.jpg)
http-graph
• Directed graph history of browsing• Reconstructs graph from referrer and URL
in the header:Referrer Request
• Lets you view your history as you took it• Shows natural “hubs” of information• See also:
http://www.uiweb.com.nyud.net:8090/issues/issue37.htm
![Page 21: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/21.jpg)
Displaying http-graph Output
• Writes a small “dot” file– “dot” part of “graphviz” tool– Use “neato” to graph– Output formats: SVG, PS, PDF, image map– Can make fully interactive!
![Page 22: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/22.jpg)
Example http-graph Output
![Page 23: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/23.jpg)
Grabbing Data with pynids
• tcp.{server, client}.data and just strings
• Any string operations will work– Searching
if “HTTP/1.0” in tcp.client.data:
– Regular Expression searchesif re.search(“HTTP/1.[10]”, tcp.client.data):
– Rewritingstring.replace(req, “GET HTTP/1.0”, “”, 1)
![Page 24: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/24.jpg)
More Fun!• Privacy invasion
– Snarf mail
• Log conversations– IRC, AIM, etc …
• Steal files– FTP, P2P apps, HTTP downloads …
• Disrupt sessionstcp.kill()
New dsniff is written in Python …
![Page 25: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/25.jpg)
flowgrep
• Marries sniffing with regular expressions
• A lot like ngrep, tcpkill, and dsniff– Logs the whole connection, not just a packet
• Look for data in streams using regular expressions
• Log or kill selected streams
• Dirt cheap IDS or IPS– Under 400 lines of code
![Page 26: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/26.jpg)
Resources
• http://www.tcpdump.org/• http://www.packetfactory.net/projects/libnids/• http://monkey.org/~provos/libevent/• http://monkey.org/~dugsong/{dpkt, pycap}• http://oss.coresecurity.com/projects/pcapy.html• http://monkey.org/~jose/software/flowgrep/• http://pilcrow.madison.wi.us/pynids/
![Page 27: Sniffing network traffic in Python Jose Nazario, Ph.D](https://reader035.vdocuments.us/reader035/viewer/2022081508/56649dd05503460f94ac4a07/html5/thumbnails/27.jpg)
Additional Resources
• Stevens, TCP/IP Illustrated vols 1 and 2
• Schiffman, Building Open Source Network Security Tools
• RFCs from the IETF