The SWIG Integration Code Generator

2017-11-03 09:05:09

But don do that. Im introducing C extension basics so you understand the underlying structure, but today, C extensions are usually better and more easily implemented with the SWIG integration code generator.

SWIG -- the Simplified Wrapper and Interface Generator -- is an open source system created by Dave Beazley. It uses C and C++ type declarations to generate complete C extension modules that integrate existing libraries for use in Python scripts. The generated C extension modules are complete: they automatically handle data conversion, error protocols, reference-count management, and more.

That is, SWIG automatically generates all the "glue" code needed to plug C and C++ components into Python programs; simply compile its output and your extension work is done. You still have to manage compilation and linking details, but the rest of the C extension task is done by SWIG.

.4.1 A Simple SWIG Example

For instance, instead of writing all that C code in the prior section, write the C function you want to use from Python without any Python integration logic at all, as though it is to be used from C alone. This is illustrated in Example 19-4.

Example 19-4. PP2EIntegrateExtendHelloLibhellolib.c

/********************************************************************* * A simple C library file, with a single function, "message", * which is to be made available for use in Python programs. * There is nothing about Python here--this C function can be * called from a C program, as well as Python (with glue code). *********************************************************************/ #include #include static char result[64]; /* this isn exported */ char * message(char *label) /* this is exported */ { strcpy(result, "Hello, "); /* build up C string */ strcat(result, label); /* add passed-in label */ return result; /* return a temporary */ }

While you e at it, define the usual C header file to declare the function externally; as shown in Example 19-5. This is probably overkill, but will prove a point.

Example 19-5. PP2EIntegrateExtendHelloLibhellolib.h

/******************************************************************** * Define hellolib.c exports to the C namespace, not to Python * programs--the latter is defined by a method registration * table in a Python extension modules code, not by this .h; ********************************************************************/ extern char *message(char *label);

Now, instead of all the Python extension glue code shown in the prior section, simply write a SWIG type declarations input file, as in Example 19-6.

Example 19-6. PP2EIntegrateExtendSwighellolib.i

/****************************************************** * Swig module description file, for a C lib file. * Generate by saying "swig -python hellolib.i". ******************************************************/ %module hellowrap %{ #include %} extern char *message(char*); /* or: %include "../HelloLib/hellolib.h" */ /* or: %include hellolib.h, and use -I arg */

This file spells out the C functions type signature. In general, SWIG scans files containing ANSI C and C++ declarations. Its input file can take the form of an interface description file (usually with an .i suffix), or a C/C++ header or source file. Interface files like this one are the most common input form; they can contain comments in C or C++ format, type declarations just like standard header files, and SWIG directives that all start with %. For example:

%module sets the modules name as known to Python importers.

%{...%} encloses code added to generated wrapper file verbatim.

extern statements declare exports in normal ANSI C/C++ syntax.

%include makes SWIG scan another file (-I flags give search paths).

In this example, SWIG could also be made to read the hellolib.h header file directly. But one of the advantages of writing special SWIG input files like hellolib.i is that you can pick and choose which functions are wrapped and exported to Python; scanning a librarys entire header file wraps everything it defines.

SWIG is really a utility that you run from your build scripts, not a programming language, so there is not much more to show here. Simply add a step to your makefile that runs SWIG, and compile its output to be linked with Python. Example 19-7 shows one way to do it on Linux.

Example 19-7. PP2EIntegrateExtendSwigmakefile.hellolib-swig

############################################################### # Use SWIG to integrate hellolib.c for use in Python programs. ############################################################### # unless youve run make install SWIG = ./myswig PY = $(MYPY) LIB = ../HelloLib # the library plus its wrapper hellowrap.so: hellolib_wrap.o $(LIB)/hellolib.o ld -shared hellolib_wrap.o $(LIB)/hellolib.o -o hellowrap.so # generated wrapper module code hellolib_wrap.o: hellolib_wrap.c $(LIB)/hellolib.h gcc hellolib_wrap.c -c -g -I$(LIB) -I$(PY)/Include -I$(PY) hellolib_wrap.c: hellolib.i $(SWIG) -python -I$(LIB) hellolib.i # C library code (in another directory) $(LIB)/hellolib.o: $(LIB)/hellolib.c $(LIB)/hellolib.h gcc $(LIB)/hellolib.c -c -g -I$(LIB) -o $(LIB)/hellolib.o clean: rm -f *.o *.so core force: rm -f *.o *.so core hellolib_wrap.c hellolib_wrap.doc

When run on the hellolob.i input file by this makefile, SWIG generates two files:

hellolib_wrap.doc is a text summary of the functions in the module.

hellolib_wrap.c is the generated C extension module glue code file.^[3]

^[3] You can wade through this generated file on the books CD (see http://examples.oreilly.com/python2) if you are so inclined. Also see file PP2EIntegrateExtendHelloLibhellolib_wrapper.con the CD for a hand-coded equivalent; its shorter because SWIG also generates extra support code.

This makefile simply runs SWIG, compiles the generated C glue code file into an .o object file, and then combines it with hellolib.c s compiled object file to produce hellowrap.so. The latter is the dynamically loaded C extension module file, and the one to place in a directory on your Python module search path (or "." if you e working in the directory where you compile).

Assuming youve got SWIG set to go, run the makefile to generate and compile wrappers for the C function. Here is the build process running on Linux:

[mark@toy ~/.../PP2E/Integrate/Extend/Swig]$ make -f makefile.hellolib-swig ./myswig -python -I../HelloLib hellolib.i Generating wrappers for Python gcc hellolib_wrap.c -c -g -I../HelloLib ...more text deleted here... ld -shared hellolib_wrap.o ../HelloLib/hellolib.o -o hellowrap.so

And once youve run this makefile, you are finished. The generated C module is used exactly like the manually coded version shown before, except that SWIG has taken care of the complicated parts automatically:

[mark@toy ~/.../PP2E/Integrate/Extend/Swig]$ python >>> import hellowrap # import the glue+library file >>> hellowrap.__file__ # cwd always searched on imports ./hellowrap.so >>> hellowrap.message(swig world) Hello, swig world

In other words, once you learn how to use SWIG, you can largely forget all the integration coding details introduced in this chapter. In fact, SWIG is so adept at generating Python glue code that its usually much easier and less error-prone to code C extensions for Python as purely C or C++-based libraries first, and later add them to Python by running their header files through SWIG, as demonstrated here.

.4.2 SWIG Details

Of course, you must have SWIG before you can run SWIG; its not part of Python itself. Unless it is already on your system, fetch SWIG off the Web (or find it at http://examples.oreilly.com/python2) and build it from its source code. Youll need a C++ compiler (e.g., g++), but the install is very simple; see SWIGs README file for more details. SWIG is a command-line program, and generally can be run just by saying this:

swig -python hellolib.i

In my build environment, things are a bit more complex because I have a custom SWIG build. I run SWIG from this csh script called myswig:

#!/bin/csh # run custom swig install source $PP2EHOME/Integrate/Extend/Swig/setup-swig.csh swig $*

This file in turn sets up pointers to the SWIG install directory by loading the following csh file, called setup-swig.csh :

# source me in csh to run SWIG with an unofficial install setenv SWIG_LIB /home/mark/PP2ndEd/dev/examples/SWIG/SWIG1.1p5/swig_lib alias swig "/home/mark/PP2ndEd/dev/examples/SWIG/SWIG1.1p5/swig"

But you won need either of these files if you run a make install command in the SWIG source directory to copy it to standard places.

Along the way in this chapter, Ill show you a few more SWIG-based alternatives to the remaining examples. You should consult the SWIG Python user manual for the full scoop, but here is a quick look at a few more SWIG highlights:

C++ "shadow" classes: Later in the chapter, Ill also show you how to use SWIG to integrate C++ classes for use in your Python scripts. When given C++ class declarations, SWIG generates glue code that makes C++ classes look just like Python classes in Python scripts. In fact, C++ classes are Python classes under SWIG; you get what SWIG calls a C++ "shadow" class that interfaces with a C++ coded extension module, which in turn talks to C++ classes. Because the integrations outer layer is Python classes, those classes may be subclassed in Python and their instances processed with normal Python object syntax.
Variables: Besides functions and C++ classes, SWIG can also wrap C global variables and constants for use in Python: they become attributes of an object named cvar inserted in generated modules (e.g., module.cvar.name fetches the value of Cs variable name from a SWIG-generated wrapper module).
Pointers: SWIG passes pointers between languages as strings (not as special Python types) for uniformity, and to allow type safety tests. For instance, a pointer to a Vector type may look like _100f8e2_Vector_p. You normally won care, because pointer values are not much to look at in C either. SWIG can also be made to handle output parameters and C++ references.
Structs: C structs are converted into a set of get and set accessor functions that are called to fetch and assign fields with a struct object pointer (e.g., module.Vector_fieldx_get(v) fetches Cs Vector.fieldx from a Vector pointer v, like Cs v->fieldx). Similar accessor functions are generated for data members and methods of C++ classes (the C++ class is roughly a struct with extra syntax), but the SWIG shadow class feature allows you to treat wrapped classes just like Python classes, instead of calling the lower-level accessor functions.

Although the SWIG examples in this book are simple, you should know that SWIG handles industrial-strength libraries just as easily. For instance, Python developers have successfully used SWIG to integrated libraries as complex as Windows extensions and commonly used graphics APIs.

SWIG can also generate integration code for other scripting languages such as Tcl and Perl. In fact, one of its underlying goals is to make components independent of scripting language choices -- C/C++ libraries can be plugged in to whatever scripting language you prefer to use (I prefer to use Python, but I might be biased). SWIGs support for things like classes seems strongest for Python, though, probably because Python is considered to be strong in the classes department. As a language-neutral integration tool, SWIG addresses some of the same goals as systems like COM and CORBA (described in Chapter 20), but provides a code-generation-based alternative instead of an object model.

You can find SWIG on this books CD (see http://examples.oreilly.com/python2) or at its home page on the Web, http://www.swig.org. Along with full source code, SWIG comes with outstanding documentation (including a manual specifically for Python), so I won cover all of its features in this book. The documentation also describes how to build SWIG extensions on Windows. A SWIG book is reportedly in the works as I write this, so be sure to check the books list at http://www.python.org for additional resources.