1. Python for CTFs
Unit 1: The Basics

2. Scripting uses in CTFs - we’ll go over each of these
Networking (Interacting with servers)
Brute Force inputs to programs
Interacting with binaries (either static or dynamic inputs that need to be fast)
Interacting with files on file system (perform operations on a bunch of files at once.

3. Before we do those, some useful Python tricks
bin(), int(), hex(), ord(), chr()
List [iterator] comprehensions
List/String converters - string.split() and “”.join(list)
Itertools
Map and Reduce - if you don’t like list comprehensions
Scripting is useful when when one needs to complete a task quickly, or when the task can be completed by combining multiple existing programs.
A caution: Scripts are often much less efficient than compiled programs, in part because scripts are usually interpreted (this adds the overhead of the interpreter) and in part because they invoke other programs (adding the overhead of program invocation).
Scripting is often used for rapid prototyping, where the goal is to build a simple version of the desired program to test different approaches and to develop a mechanism for testing user interfaces. One can quickly implement different algorithms and try different data representations to find the one that is most suitable (due to performance, interaction with other components, user needs, and so forth).

4. bin(), int(), hex(), ord(), chr()
bin(intstring) - returns binary string representing the integer (kind of weird, will need to read up on it before use, different in python2 and 3)
int(intstring, n) - returns the base 10 integer representation of intstring as a base n number (as an integer)
hex(intstring) - returns hex representation of an integer
ord(character) - returns ascii value of that character
chr(int) - converts int to ascii value

5. Examples
We will use the Bourne shell command language to develop a Bourne shell script (called a shell script from this point on). This shell, or command interpreter, is universal on all Linux systems as well as all versions of UNIX such as FreeBSD, OpenBSD, and so forth.
This language allows one to combine programs already written to perform a task or a set of tasks. It also provides a framework for testing conditions and performing I/O both through the command line and using files.
A later version of this shell, called the Bourne Again SHell or BASH, provides a superset of this language, including adding functions. We will not use these because on systems without Bash, the functions may not work.
Although this language is particular to the Bourne shell, the ideas and many of the constructs also apply to other *nix command interpreters such as the C shell (csh and tcsh), the Korn shell, and so forth.

6. List comprehensions
Create a list (or tuple/dict) by performing some operation on elements of an iterator:
It’s probably a good idea here to review the hierarchical Linux file structure. The /bin directory contains many system programs including the Bourne shell (sh).
Also, emphasize that the file is simply a text file. The problem with Microsoft Word and word processing programs is that they include formatting commands that mess up the interpretation of the script. Saving a word processing file as text probably won’t work, as these programs often combine lines or add line breaks at odd locations. It is best to use vi (1), emacs (1), or some editor that saves what you type as ASCII text.
Technically, you can name the file using almost any ASCII character, but some are metacharacters (like ‘*’, ‘{‘, and blank) and so need to be escaped. Metacharacters are explained in a later part, so for now it’s simplest to have students follow the rule of letters and numbers only.
These presentation notes follow the Linux/UNIX convention of giving the section of the manual with the description of the command in parentheses. For example, grep and sed are in Section 1 of the manual, so they are written "grep (1)" and "sed (1)" the first time they appear.

7. string.split() and “”.join(list)
string.split() takes a character delimiter as a parameter and uses this to partition the string into a list
“”.join(list) takes each element of the list and concatenates with what is inside the string
This assumes the current working directory contains the script “abcscript”, which is provided in 04.SeS_Unit1_TheBasics_DataFiles.
In general, in shell scripts, lines that begin with “#” are comments and are ignored. That’s why the first line does not cause problems with this method.

8. Examples
It’s worth discussing the chmod (1) command. This command line sets the user rights to read, write, and execute, and everyone else’s rights to read and execute (“g” stands for “group” and “o” for “other”).
If you set up the search path so it contains the current working directory “.”, you can simply type “abcscript”. Warning: This is very bad security practice, so I emphatically do not recommend it! But in all the examples, I omit the “./” and/or the “sh”.
What actually happens when you execute the script using Method 2:
The kernel opens the file and loads the first two bytes into memory. These are “#!”.
The kernel recognizes this as a “magic number” and reads the rest of the line. This is the name of the program that will be used to interpret the script.
It then executes the program, giving it the name of the script and any arguments as parameters. It also sets things up so the name of the script is argument 0, and so forth.
If you omit the first line, some systems will simply assume you meant the Bourne shell and act as though you put it there. Others will give an error. It’s good practice to put it in (that is, robust scripts have this).
In general, in shell scripts, lines that begin with “#” are comments and are ignored. This is the only exception.

9. Itertools
Useful for getting permutations/combinations of lists and other iterables
Most useful: combinations and permutations
combinations(iterable, n) - returns all possible position-dependent combinations of length n from the iterable (i.e. if ‘ab’ is there,
‘ba’ won’t be and converse)
permutations(iterable, n) - returns all position-dependent combinations of length n from the iterable (i.e. if ‘ab’ is there, ‘ba’ will be and converse)
Commenting in scripts is like commenting in programs:
Comments should say what the code is doing at a high level, not how it is doing it—that should be clear from the code.
If the code is obscure, tricky, or weird, then it is appropriate to explain how the code does it.
At all costs avoid comments like “Add 1 to x” or “Assign the variable X the value ‘yes’.”

10. Map and reduce
map(fcn,iterable) is very similar to list comprehensions, uses each element of the iterable as argument to fcn() and returns results as a list
reduce(fcn, iterable) performs an operation (such as addition or multiplication) on the entire iterable and returns the result
map is almost identical to a list comprehension, use your preference of the syntax/whatever you think is most readable
reduce is equivalent to a running counter/sum/result in a for loop
The two commands here emphasize that # must be preceded by white space or it will be considered part of the word in which it occurs.
Indeed, the students will see the variable reference “$#” (it means the number of arguments to the script).

11. Examples
The material on this slide and the next slide (Slides 11-12) is basic; students have certainly seen something like it in their first programming class.
The only point that may be new to them is the emphasis on integrating robustness (error checking and such) into Steps 3, 4, and 5, rather than writing the program and then adding security on later.

12. Networking How to connect to servers
Many ways to do it in python, including sockets and higher levels, we will use telnetlib for now
This is just to understand the basics. Later we will use pwntools
“Corner cases” are cases that use data at or crossing limits. For example, if a script requires a command-line argument, try not giving it any or give it two, then see if the script handles those cases reasonably. Here, “reasonably” must be interpreted in light of the requirements, the environment, and so forth.

13. Telnetlib key ideas Declare telnetlib object
Read with read_all() or read_until()
Write with write()
This problem corresponds to Lab Exercise 1 in 03.SS_Unit1_TheBasics_Lab.
Here, we state the problem and start breaking it down along the lines discussed earlier.
First, what is a word? One common definition is any nonempty sequence of alphabetic characters. But does that include numbers, apostrophes, and hyphens? That depends on interpretation. So let’s look at the data and see if it suggests anything.

14. Brute Forcing Perform operations over iterators
For best performance, use a generator function and/or a faster-performance interpreter (Pypy or Cython)
Don’t use nested for loops
Bad performance
Use itertools or list comprehensions
This slide corresponds to Lab Exercise 1, Part A, in 03.SS_Unit1_TheBasics_Lab.
Looking at the file, it’s clear the file is a list of words, one per line, so we just say that a word is whatever is on a single line.
This highlights the need to understand the data to be analyzed. Here, text files generally have multiple words per line. On those files, you would need to use a completely different tactic. But with one word per line, the problem is much easier to solve.

15. Interacting with binaries
Two choices:
import os; os.system(‘cmd’)
import subprocess; subprocess.call/check_output
Use check_ouput to get the output from commands
This slide corresponds to Lab Exercise 1, Part B, in 03.SS_Unit1_TheBasics_Lab.
Note that you can use a pattern to express a character sequence. Here it’s easy as there are no metacharacters to be matched.
A useful tool for finding commands is the command apropos (1):
Try apropos match.
Look down the output for something that says “print lines matching” or something similar and has a “(1)” on it; the (1) means it’s a user command.

16. Example
Others that work are sed (1) and awk (1), but these are more complicated, so we won’t use them.
If looking for the command will distract students too much, you can simply say “We will use grep to look for the pattern ‘gry’.”

17. Interacting with binaries
Can also just use single string command
Here, emphasize that this is an existing command (they do not need to write it), and that “pattern” here can also be a character sequence like “gry”.
The “more that we’ll get to later” will come up in Unit 2, where we do use a real pattern.

18. Interacting with files
You can read/write to/from files with the open(filename, ‘r’|’w’) call
Or use subprocess and format strings
This slide corresponds to Lab Exercise 1, Part C, in 03.SS_Unit1_TheBasics_Lab.
This repeats what we said earlier, and re-emphasizes how the interpreter works. In the command, the “/bin/” says where the shell is, and the “sh” names the command.
Similarly, in the second example, the Python interpreter, called “python”, is located in the directory “/usr/bin”.

19. Homework Exercise 1 Coins - nc <ctf_server_ip> 8889
Figure this one out!
● Robots - nc <ctf_server_ip> 9998
Robots have encoded messages by indexing into a string of characters and performing an xor then shift on that index.
This is Lab Exercise 1, Part C, in 03.SS_Unit1_TheBasics_Lab.
Step 5 of the approached outlined on Slides 11-12, “Test it,” is included in Part C of Lab Exercise 1. We’re doing only basic testing. Part C of Lab Exercise 1 says what the script should print. Here are the words:
agrypnia
agrypnode
arthrogryposis
grylle
grypanian
gryph
gryposis
puggry