stuff

Extending Python with C/C++ (part 1)

July 10, 2010

I've been researching ways to write modules in C (and C++). Using plain Python/C API is tedious and repetitive, but fortunately there are better options (quite many, I might add).

Let's start with ctypes, a foreign function library for Python; it allows you to call functions in shared libraries without writing any additional C/C++ code yourself. It comes bundled with Python since version 2.5 and it doesn't require you to recompile C/C++ code, which makes it a pretty good choice for the job; but it doesn't handle C++ features - e.g. classes. Also, once you're creating modules a bit more complex than a few simple functions, you need to understand various features in C, especially memory management.

An example. I decided it would be fun to write an implementation of bogosort. Here's a pure Python version:

from random import shuffle
from itertools import izip

def bogosort(int_array):
  while not all(x <= y for x,y in izip(int_array, int_array[1:])):
    shuffle(int_array)

a = [4,1,3,2,5]
bogosort(a)
print a

Just shuffling the list and checking if it's sorted:) Here's the C++ reimplementation:

#include <cstdlib>
#include <ctime>

void shuffle(int *array, int length)
{
  for (int i = 0; i < length-1; i++)
  {
    int j = rand() % (length-i-1) + i+1;
    int t = array[i];
    array[i] = array[j];
    array[j] = t;
  }
}

bool not_sorted(int *array, int length)
{
  for (int i = 0; i < length-1; i++)
  {
    if (array[i] > array[i+1]) return true;
  }
  return false;
}

extern "C" void bogosort(int *array, int length)
{
  srand(time(NULL));
  while (not_sorted(array, length))
  {
    shuffle(array, length);
  }
}

Adding extern "C" prevents name mangling for the function I want to export (actually, all three functions above are exported by default as far as GCC is concerned; just names of shuffle and not_sorted are mangled). Let's compile this code into a shared library libbogo.so and check exported symbols:

g++ -fPIC -g -c -Wall bogosort.cpp
g++ -shared -Wl,-soname,libbogo.so -o libbogo.so bogosort.o
nm -gD --defined-only libbogo.so

The output should be something like this:

0000067b T _Z10not_sortedPii
000005ec T _Z7shufflePii
00001948 A __bss_start
00001948 A _edata
00001950 A _end
00000758 T _fini
00000480 T _init
000006c9 T bogosort

It's time for ctypes:

from ctypes import CDLL, c_int

def bogosort(int_array):
  # load the library and access the function,
  # store it for subsequent calls
  if not hasattr(bogosort, 'bogosort_raw'):
    bogosort_raw = CDLL('./libbogo.so').bogosort
    setattr(bogosort, 'bogosort_raw', bogosort_raw)
  else:
    bogosort_raw = getattr(bogosort, 'bogosort_raw')
  # passing a list doesn't work, we need to create
  # a proxy object
  ia = (c_int * len(int_array))(*int_array)
  bogosort_raw(ia, len(ia))
  return list(ia) # reconstruct usual python list

a = [4,2,5,1,3,6,9,7]
a = bogosort(a)
print a

Only None, integers and strings can be used as parameters in exported functions. In order to pass a list of integers to the function we create an array type (by multiplying c_int with length of the array) and instantiate it (unpacking array into arguments).

Part 2 coming soon