Python C API overview References:

Two uses of the Python/C API 1.Extending – Write a dll (.pyd), which exports an initialization function a list of "things" in the module – classes – functions – variables – You can import this just like any (pure) python module – Use the c types / functions in your program import mymodule mymodule.fun(4.1) # A c-function – Many python modules are built this way math pygame numpy …

Two uses of the Python/C API 2.Embedding – Create a python interpreter inside a C program (executable) – Can execute python code Usually from a file Possibly from strings hard-coded, generated by the program. – Need to convert to/from python types – Usually interacts with other C/C++ constructs in the executable.

The Python / C API A ton of functions. Used in both extensions and embedding. Where are they? – Prototypes (.h) c:\pythonXY\include – Code (.lib) c:\pythonXY\libs\python32.lib – Dll (.dll) c:\pythonXY\DLLs\python3.dll Only necessary if embedding Debug vs. Release – The pre-built (windows) binary only include Release builds – If you want Debug dll's and.lib's you have to build from source.

API Naming convention PyXXX_YYY – (Python) class XXX, functionYYY – e.g. PyTuple_Size(…) Py_ZZZ – Global function ZZZ – e.g. Py_Compile(…)

PyObject's Literally everything in Python is a PyObject* – Int ref-count (more on this shortly) – A type structure – Any other data e.g. for tuples, they have a length attribute. The python interpreter owns all objects – The user just has references to them – The first time it is referenced, python malloc's it. – When the refcnt reaches 0, it eventually gets free'd – In (normal) python this is done for us… – …but in the C API we have to manage it.

RefCounting Take a simple example def foo(a): return a[0] ** 2 + a[1] ** 2 x = [3, 2] y = [foo(x), 3.2] z = y z = None Line 1 – 2 creates a new function object. Line 3 creates a new list x. Line 4 calls foo. o a and x refer to the same list (refcnt is 2) o Creates another float (13.0) and returns it o a goes out of scope – the refcnt on the list is now 1 o The 13.0 and 3.2 are part of a new list object (refcnt = 1) Line 5 makes a second reference to the list (Refcnt = 2) Line 6 decreases the refcnt on the list.

RefCounting, cont. …In the C API, you have to manually do it. – Memory Leaks (if you forget to decrement a ref-count) – Invalid memory access (if you forget to increment, and then the object is destroyed) Three functions Py_INCREF(PyObject*); Py_DECREF(PyObject*); Py_XDECREF(PyObject*); // Like previous, but has a NULL- check

When to modify Ref-cnts Look in the C/API documentation – New Reference e.g. PyLong_FromLong(long v) You own a ref-count to it. Make sure to call Py_DECREF when done. – Borrowed Reference e.g. PyList_GetItem(PyObject * list, Py_ssize_t index) Someone else holds a reference Don't call DECREF (unless you called INCREF on it) – Stolen Reference e.g. PyTuple_SetItem(PyObject * tuple, Py_ssize_t index, PyObject * item); You usually have created the PyObject item and own a reference to it. After calling the above func, the tuple stole your reference to item, so don't call DECREF on it.

Converting / Creating variables Integers – C => Python PyObject * PyLong_FromLong(int i); A new reference – make sure you DECREF it! – Python => C int PyLong_AsLong(PyObject*); float PyLong_AsDouble(PyObject*); … – Determining if a PyObject is a (python) int int PyLong_Check(PyObject*); // 1 if it is, 0 if not

Converting / Creating variables, cont. Floats – C => Python PyObject * PyFloat_FromDouble(double d); A new reference – make sure you DECREF it! – Python => C double PyFloat_AsDouble(PyObject*); – Determining if a PyObject* is a (python) float int PyFloat_Check(PyObject*); // 1 if it is.

Converting / Creating variables, cont. Strings – A bit more complicated because python 3.x uses unicode internally, not ANSI (char*) strings. – C => Python PyObject * PyUnicode_FromString(char * s); A new reference – make sure you DECREF it! – Python => C char temps[256]; // PObj is the object (in python) we want to convert PyObject * pBytesObj = PyUnicode_AsUTF8String(pObj); if (pBytesObj) { strcpy(temps, PyBytes_AsString(pBytesObj)); Py_DECREF(pBytesObj); } – Determining if a PyObject* is a (python) unicode object. int PyUnicode_Check(PyObject*); // 1 if it is.

Tuples Determining if a PyObject is a tuple int PyTuple_Check(PyObject*); // 1 if it is. Creating a new tuple PyObject * PyTuple_New(Py_ssize_t len); – Inserting objects into a new tuple PyTuple_SetItem( PyObject * tup, Py_ssize_t index, PyObject * item ); – Note: this steals a reference to item Getting the size of a tuple Py_ssize_t PyTuple_Size(PyObject * tup); Getting an element of a tuple PyOBject * PyTuple_GetItem(PyOBject * tup, Py_ssize_t indx); – Note: this borrows a reference to the item (i.e. the tuple still owns a reference, so don't DECREF if)

Raising exceptions Something happened in your C++ code and you want to raise a python exception If this is in a method / function, you usually return NULL. It is also a good idea to raise an exception so the (python) user can see what happened. PyErr_SetString(PyExc_ValueError, "You did … wrong."); There a bunch of other PyExc_XYZ objects

Handling Exceptions Often a (python API) function fails. – It returns NULL (not helpful) – If usually also sets an exception (helpful) To retrieve the error: void PyErr_Fetch(PyObject ** pType, PyObject ** pValue, PyOBject ** pTraceback); – Note: You need to pass the address of a PyOBject* (use the ampersand) – this function will fill them in. To convert pValue (the error string) to a C-string, use: PyObject * PyObject_Str(myPValue); // New ref – Then convert this (unicode) object to a char * as before.

C/C++ extensions IDLE (or other python interpreter) Python code (.py) C/C++ extension (a.pyd file). Can contain: Python interfaces to C "classes" Python interfaces to C functions Python interfaces to C variables

Creating a C extension (pyd) Goal: – A new class (written in C) – A new function (written in C) – A variable (written in C) – Access all 3 in python Example.py file (myCMod is a C extension): import myCMod C = myCMod.myClass("Bob") print(C.name) # Bob print(C.timestamp) # 0 C.incTimestamp() print(C.timestamp) # 1 print(myCMod.fact(5)) # 120 print(myCMod.aVariable) # Surprise!

Making an extension (pyd) 0. Set up a project – A Win32 console dll (empty) – Put all your code in a single.cpp file – Change output to xyz.pyd (actually a.dll) – Only do a release build (debug can only be run by a debug build of python) – Include includes / libs paths – Link with python32.lib Reference:

Making an extension (pyd) 1.Create a C-type ("guts" of a python object) 2.Create methods for the new type – static PyObject * asdfasdf(…) 3.Create an attribute (PyMemberDef) and method (PyMethodDef) structure for the new type 4.Create a "dictionary" (static PyTypeObject) of functions and attributes for the new type 5.Create a C function in the module (factorial) – I'll let you experiment with this 6.Create a C variable in the module – I'll let you experiment with this too… 7.Make a PyModuleDef to package the module 8.Make a "main" function PyMODINIT_FUNC PyInit_mycmod(void) – Note: in place of mycmod, put the name of the pyd file (exactly)

Embedding the Interpreter C/C++ Program (.exe) Python DLL Python Interpreter Python code (strings) Python (.py) files Other C/C++ objects

Creating the embedded interpreter First, Add any internal C modules – These look just as they did for an extension We aren't building a.dll – they exist solely within the executable Includes: – Define any C classes with Python interfaces – Defining the C methods (return values of PyObject *) – Defining the PyMethodDef structure for the class – Define the PyMethodDef for any stand-alone functions – Define the PyModuleDef structure.

Creating the embedded interpreter "Import" the C module into the embedded interpreter: PyImport_AppendInittab(char * modName, PyObject * (*funcPtr)(void)); The second parameter is a function pointer Pass the address of a function created as in the last step of making an extension. – This function creates an returns a (Python) module object – Possibly also adds classes to this module (if applicable) The function should have a prototype of PyObject * PyInit_myMod(void)

Creating the embedded interpreter Second, start up the interpreter Py_Initialize() – Note: this interpreter will (only) have modules defined with PyImport_AppendInittab. – To add other modules…??? Third, run the rest of the program – At appropriate times, load and run.py files into the embedded interpreter (more on this shortly) Finally, shut down the interpreter Py_Finalize()

Loading / Executing a.py file Load the file into a char * buffer – FILE* of fstream and new / delete Compile the buffer into a code module PyObject * Py_CompileString(char * buff, char * fname, Py_file_input) Import and Execute the code module PyImport_ExecCodeModule(char * name, PyObject * codeMod); – This will run the python code. – If that code references the C functions / classes with Python interfaces, that code will now be run. This will usually interact with the rest of the C structures in the program