How to write a Python Extension

• Creating a Python Module
• Creating a Python type
• Organizing your C Code
• Documentation strings

Some hints

• As much as possible, make the top-level symbols static. It is useless to populate the global C name-space by exporting symbols needlessly.
• Use forward declarations judiciously. C is a flexible language, even strict ANSI C, when written appropriately.
• Try to write the functionality as simply as possible. Design the Python interface to perform specific operations. This will allow you to write a more flexible wrapper module on top of the C module (cf. Tkinter vs. _tkinter).

The bare minimum

To create a module, you need a name and an initialization routine, to be called by Python.

PyMethodDef methods[] = {
  {NULL, NULL},
};

void initmodname()
  {
    (void)Py_InitModule("modname", methods);
  }

Adding a function

Create a function that takes Python objects and returns a pointer to a Python object.

PyObject *MyCommand(self, args)
  PyObject *self, *args;
  { PyObject *result = NULL;
    long a, b;

    if (PyArg_ParseTuple(args, "ii", &a, &b)) {
      result = Py_BuildValue("i", a + b);
    } /* otherwise there is an error,
       * the exception already raised by PyArg_ParseTuple, and NULL is
       * returned.
       */
    return result;
  }

PyMethodDef methods[] = {
  {"add", MyCommand, METH_VARARGS},
  {NULL, NULL},
};

To add a function later, edit the methods array inserting the line. Since the module name-space is a dictionary, the order of the entries is up to the coder's best judgment.

Adding "constants"

You added a constant by creating a Python object and binding it to the module name-space. Since the module global name-space is a dictionary, modification is the same way.

void initmodname()
  { PyObject *m, *d;
    PyObject *tmp;

    m = Py_InitModule("modname", methods);
    d = PyModule_GetDict(m);

    tmp = PyFloat_AsDouble(3.1415926);
    PyDict_SetItemString(d, "pi", tmp);
    Py_DECREF(tmp);
  }

Making an exception

Often you will want to make an exception type for your module.

PyObject *exception = NULL;

...

void initmodname()
  { PyObject *m, *d;
    ...
#if PYTHON_API_VERSION >= 1007
    exception = Py_NewException("modname.error", NULL, NULL);
#else
    exception = Py_BuildValue("s", "modname.error");
#endif
    PyDict_SetItemString(d, "error", exception);
  }

At this point, you can raise exceptions with the normal Python API.

  PyErr_SetString(exception, "I could not do that");

For the sake of explanation, I will build a small (fairly useless) object that autoincrements and emulates a number otherwise.

The pieces

• Object structure
• Type object definition
• Helper macros
• Creation and destruction
• Defining data attributes
• Defining methods
• Emulating Numbers
• Emulating Sequences
• Emulating Mappings
• Emulating Callables
• Comparing objects
• Printing an object

One important point: You will not want to emulate both a sequence and a mapping. There are overlapping callback methods between the two. The more generic is a mapping, and the implementation is still up to the coder.

The Python object structure

Create normal C structure with the exception of a Python header, by including PyObject_HEAD.

typedef struct {
  PyObject_HEAD
  long value;
} counter;

The Python type object definition

Just about all the Python types follow this form. It defines the behavior of the type; for example, do we treat it as a dictionary?

PyTypeObject counter_Type = {
  PyObject_HEAD_INIT(&PyType_Type)
  0,
  "counter",                /* char *tp_name; */
  sizeof(counter),          /* int tp_basicsize; */
  0,                        /* int tp_itemsize;       /* not used much */
  counter_dealloc,          /* destructor tp_dealloc; */
  counter_print,            /* printfunc  tp_print;   */
  counter_getattr,          /* getattrfunc  tp_getattr; /* __getattr__ */
  counter_setattr,          /* setattrfunc  tp_setattr;  /* __setattr__ */
  counter_compare,          /* cmpfunc  tp_compare;  /* __cmp__ */
  counter_repr,             /* reprfunc  tp_repr;    /* __repr__ */
  &counter_as_number,       /* PyNumberMethods *tp_as_number; */
  0,                        /* PySequenceMethods *tp_as_sequence; */
  0,                        /* PyMappingMethods *tp_as_mapping; */
  counter_hash              /* hashfunc tp_hash;     /* __hash__ */
  0,                        /* ternaryfunc tp_call;  /* __call__ */
  counter_str,              /* reprfunc tp_str;      /* __str__ */
};

Helper macros

It is often useful to create C macros that access the object inside the module.

#define Counter_Check(v)  ((v)->ob_type == &counter_Type)
#define Counter_value(v)  (((counter *)(v))->value)

Creation and destruction

PyObject *
counter_NEW(initial_value)
  long initial_value;
  { counter *object = NULL;
    
    object = PyObject_NEW(counter, &counter_Type);
    if (object != NULL)
      object->value = initial_value;
    return (PyObject *)object;
  }

void
counter_dealloc(self)
  PyObject *self;
  {
    PyMem_DEL(self);
  }

/* this is the Python constructor, as opposed to the C constructor */
/* this is a normal module function */
PyObject *
counter_new(self, args)
  PyObject *self, *args;
  { PyObject *object = NULL;
    long value = 0;  /* default initial value */

    if (PyArg_ParseTuple(args, "|i", &value))
      object = counter_NEW(value);
    return object;
  }

The destructor is registered in the tp_dealloc slot of the PyTypeObject definition (above). When the reference count goes to zero, the destructor is called.

Defining data attributes

PyObject *counter_getattr(self, attrname)
  PyObject *self;
  char *attrname;
  { PyObject *result = NULL;
    counter *object = (counter *)self;
    
    if (strcmp(attrname, "value") == 0)
      result = PyInt_AsLong(object->value);
    else
      PyErr_SetString(PyExc_AttributeError, attrname);
    return result;
  }

/* the C setattr function should return an integer, 0 on success, -1
   when an exception is raised.  When the given value is NULL, then
   "delattr" was called instead of "setattr".
   (Remember Python's None is Py_None, not NULL)
 */
int counter_setattr(self, attrname, value)
  PyObject *self, *value;
  char *attrname;
  { int result = -1;
    counter *object = (counter *)self;
    long newval;

    /* we will let the user reset the value, just for show */
    if (strcmp(attrname, "value") == 0) {
      if (value != NULL) {
        if (PyArg_Parse(value, "i", &newval)) {
          object->value = newval;
          result = 0;  /* success */
        } /* an exception is raised */
      } else /* a `del obj.value' statement */
        PyErr_SetString(PyErr_TypeObject, "read-only attribute");
    } else
      PyErr_SetString(PyExc_AttributeError, attrname);
    return result;
  }

Defining methods

Defining a method is a combination of defining a module function (which uses its self argument) and returning a data attribute.

PyObject *
counter_incr(self, args)
  PyObject *self, *args;
  { PyObject *result = NULL;
    counter *object = (counter *)self;

    if (PyArg_ParseTuple(args, "")) {  /* takes no args, other than self */
      object->value++;
      result = Py_None;
      Py_INCREF(result);
    }
    return result;
  }

PyMethodDef counter_methods[] = {
  {"incr", counter_incr, METH_VARARGS},
  {NULL, NULL},
};

PyObject *counter_getattr(self, attrname)
  PyObject *self;
  char *attrname;
  { PyObject *result = NULL;
    counter *object = (counter *)self;
    
    if (strcmp(attrname, "value") == 0)
      result = PyInt_AsLong(object->value);
    else /* find and return a bound method */
      result = Py_FindMethod(counter_methods, self, attrname);
    return result;
  }

Emulating Callables

There may be times when you want to make an object callable. The method is similar for normal functions, but the keyword arguments must be handled (which are only optional with normal functions).

PyObject *
counter_call(self, args, keywds)
  PyObject *self, *args, *keywds;
  { PyObject *result = NULL;
    counter *object = (counter *)self;
    static char *kwlist[] = { NULL };  /* no keywords */

    /* check that there are no passed arguments or keywords */
    if (keywds == NULL ?
        PyArg_ParseTuple(args, "") :
        PyArg_ParseTupleAndKeywords(args, keywds, "", kwlist))
      PySys_WriteStdout("The value of the counter is %d.\n", object->value);
      result = Py_None;
      Py_INCREF(result);
    }
    return result;
  }

Comparing objects

int counter_compare(self, other)
  PyObject *self, *other;
  { int result = 0;
    counter *object = (counter *)self;
    PyObject *tmp;

    tmp = PyInt_FromLong(object->value);
    result = PyObject_Compare(tmp, other);
    Py_DECREF(result);
    return result;
  }

long
counter_hash(self)
  PyObject *self;
  { counter *object = (counter *)self;
    PyObject *temp;
    long result = -1

    temp = PyInt_FromLong(object->value);
    if (temp == NULL)
      result = PyObject_Hash(temp);
    Py_XDECREF(temp);
    return result;
  }

Printing an object

There are three methods of printing an object. They relate to print a, repr(a) and str(a)

int counter_print(self, fp, flags)
  PyObject *self;
  FILE *fp;
  int flags; /* can be ignored in Python 1.5.x and earlier */
  { counter *object = (counter *)self;
    fprintf(fp, "%d", object->value);
    return 0;
  }

PyObject *counter_repr(self)
  PyObject *self;
  { counter *object = (counter *)self;
    char buf[100];
    sprintf(buf, "<counter %d>", object->value);
    return PyString_FromString(buf);
  }

PyObject *counter_str(self)
  PyObject *self;
  { counter *object = (counter *)self;
    char buf[100];
    sprintf(buf, "%d", object->value);
    return PyString_FromString(buf);
  }

1. Headers
2. Module globals (exceptions, thread locks, etc.)
3. Module forward declarations
4. Support routines
5. For each type
   1. Type forward declaration
   2. Object definition
   3. Helper macros
   4. Helper functions
   5. C constructor and destructor
   6. Emulation routines (including the *_as_* structure)
   7. Methods
   8. Method table
   9. Callback routines (__str__, __getattr__, etc.)
   10. Type definition
   11. Python constructor
6. Module functions
7. Module init function (initmodname())

With the exception of "initmodname()", just about every other symbol can be declared static.

Example code

Method prototypes

Here is a list of the prototypes used in type and method definitions. This should help you write the appropriate functions yourself.

void (*destructor) (PyObject *self);
int (*printfunc) (PyObject *self, FILE *fp, int flags);
PyObject *(*getattrfunc) (PyObject *self, char *attrname);
int (*setattrfunc) (PyObject *self, char *attrname, PyObject *value);
int (*cmpfunc) (PyObject *v, PyObject *w);
PyObject *(*reprfunc) (PyObject *self);
long (*hashfunc) (PyObject *self);
int (*inquiry) (PyObject *self);
int (*coercion) (PyObject **v, **w);
PyObject *(*intargfunc) (PyObject *self, int i);
PyObject *(*intintargfunc) (PyObject *self, int il, int ih);
int (*intobjargproc) (PyObject *self, int i, PyObject *value);
int (*intintobjargproc) (PyObject *self, int il, int ih, PyObject *value);
int (*objobjargproc) (PyObject *self, PyObject *i, PyObject *value);
PyObject *(*unaryfunc) (PyObject *self);
PyObject *(*binaryfunc) (PyObject *self, PyObject *obj);
PyObject *(*ternaryfunc) (PyObject *self, PyObject *args, PyObject *keywds);