ppembed: A High-Level Embedding API
But don do that . As you can probably tell from the last example, embedded-mode integration code can very quickly become as complicated as extending code for nontrivial use. Today, no automation solution solves the embedding problem as well as SWIG addresses extending. Because embedding does not impose the kind of structure that extension modules and types provide, its much more of an open-ended problem; what automates one embedding strategy might be completely useless in another.
With a little up-front work, though, you can still automate common embedding tasks by wrapping up calls in a higher-level API. These APIs could handle things such as error detection, reference counts, data conversions, and so on. One such API, ppembed, is available on this books CD (see http://examples.oreilly.com/python2). It merely combines existing tools in Pythons standard C API to provide a set of easier-to-use calls for running Python programs from C.
.6.1 Running Objects with ppembed
Example 20-15 demonstrates how to recode objects-err-low.c by linking ppembeds library files with your program.
Example 20-15. PP2EIntegrateEmbedApiClientsobject-api.c
#include This file uses two ppembed calls (the names that start with
"PP") to make the class instance and call its method.
Because ppembed handles error checks, reference counts, data
conversions, and so on, there isn much else to do here. When
this program is run and linked with ppembed library code, it works
like the original, but is much easier to read, write, and debug:
[mark@toy ~/.../PP2E/Integrate/Embed/ApiClients]$ objects-api
brave sir robin
The ppembed API provides
higher-level calls for most of the embedding techniques weve
seen in this chapter. For example, the C program in Example 20-16 runs code strings to make the
string module capitalize a simple text.
#include /* standard API defs */
void error(char *msg) { printf("%s
", msg); exit(1); }
main( ) {
/* run strings with low-level calls */
char *cstr;
PyObject *pstr, *pmod, *pdict; /* with error tests */
Py_Initialize( );
/* result = string.upper(spam) + ! */
pmod = PyImport_ImportModule("string"); /* fetch module */
if (pmod == NULL) /* for name-space */
error("Can import module");
pdict = PyModule_GetDict(pmod); /* string.__dict__ */
Py_DECREF(pmod);
if (pdict == NULL)
error("Can get module dict");
pstr = PyRun_String("upper(spam) + !", Py_eval_input, pdict, pdict);
if (pstr == NULL)
error("Error while running string");
/* convert result to C */
if (!PyArg_Parse(pstr, "s", &cstr))
error("Bad result type");
printf("%s
", cstr);
Py_DECREF(pstr); /* free exported objects, not pdict */
}
This C program file includes politically correct error tests after
each API call. When run, it prints the result returned by running an
uppercase conversion call in the namespace of the Python
string module:
[mark@toy ~/.../PP2E/Integrate/Embed/ApiClients]$ codestring-low
SPAM!
You can implement such integrations by calling Python API functions
directly, but you don necessarily have to. With a
higher-level embedding API like ppembed, the task can be noticeably
simpler, as shown in Example 20-17.
#include "ppembed.h"
#include When linked with the ppembed library code, this version produces the
same result as the former. Like most higher-level APIs, ppembed makes
some usage mode assumptions that are not universally applicable; when
they match the embedding task at hand, though, such wrapper calls can
cut much clutter from programs that need to run embedded Python code.
Embedded Python code can do useful
work as well. For instance, the C program in Example 20-18 calls ppembed functions to run a string of
Python code fetched from a file that performs validation tests on
inventory data. To save space, Im not going list all the
components used by this example (though you can find them at http://examples.oreilly.com/python2). Still, this file shows the embedding portions
relevant to this chapter: it sets variables in the Python
codes namespace to serve as input, runs the Python code, and
then fetches names out of the codes namespace as
results.[8]
[8] This is more or less the kind of structure
used when Python is embedded in HTML files in the Active Scripting
extension, except that the globals set here (e.g.,
PRODUCT) become names preset to web browser
objects, and the code is extracted from a web page, not fetched from
a text file with a known name. See Chapter 15.
/* run embedded code-string validations */
#include
#include There are a couple of things worth noticing here. First of all, in
practice this program might fetch the Python code files name
or path from configurable shell variables; here, it is loaded from
the current directory. Secondly, you could also code this program by
using straight API calls instead of ppembed, but each of the
"PP" calls here would then grow into a chunk of more
complex code. As coded, you can compile and link this file with
Python and ppembed library files to build a program. The Python code
run by the resulting C program lives in Example 20-19;
it uses preset globals and is assumed to set globals to send result
strings back to C.
# embedded validation code, run from C
# input vars: PRODUCT, QUANTITY, BUYER
# output vars: ERRORS, WARNINGS
import string # all python tools are available to embedded code
import inventory # plus C extensions, Python modules, classes,..
msgs, errs = [], [] # warning, error message lists
def validate_order( ):
if PRODUCT not in inventory.skus( ): # this function could be imported
errs.append(ad-product) # from a user-defined module too
elif QUANTITY > inventory.stock(PRODUCT):
errs.append(check-quantity)
else:
inventory.reduce(PRODUCT, QUANTITY)
if inventory.stock(PRODUCT) / QUANTITY < 2:
msgs.append(
eorder-soon: + `PRODUCT`)
first, last = BUYER[0], BUYER[1:] # code is changeable on-site:
if first not in string.uppercase: # this file is run as one long
errs.append(uyer-name: + first) # code-string, with input and
if BUYER not in inventory.buyers( ): # output vars used by the C app
msgs.append(
ew-buyer-added)
inventory.add_buyer(BUYER)
validate_order( )
ERRORS = string.join(errs) # add a space between messages
WARNINGS = string.join(msgs) # pass out as strings: "" == none
Don sweat the details in this code; some components it uses
are not listed here either (see http://examples.oreilly.com/python2 for the full
implementation). The thing you should notice, though, is that this
code file can contain any kind of Python code -- it can define
functions and classes, use sockets and threads, and so on. When you
embed Python, you get a full-featured extension language for free.
Perhaps even more importantly, because this file is Python code, it
can be changed arbitrarily without having to recompile the C program.
Such flexibility is especially useful after a system has been shipped
and installed.
As discussed earlier, there is a variety of ways to structure
embedded Python code. For instance, you can implement similar
flexibility by delegating actions to Python
functions fetched from
module files, as illustrated in Example 20-20.
/* run embedded module-function validations */
#include
#include The difference here is that the Python code file (shown in Example 20-21) is imported, and so must live on the Python
module search path. It also is assumed to contain functions, not a
simple list of statements. Strings can live anywhere -- files,
databases, web pages, and so on, and may be simpler for end users to
code. But assuming that the extra requirements of module functions
are not prohibitive, functions provide a natural communication model
in the form of arguments and return values.
# embedded validation code, run from C
# input = args, output = return value tuple
import string
import inventory
def validate(product, quantity, buyer): # function called by name
msgs, errs = [], [] # via mod/func name strings
first, last = buyer[0], buyer[1:]
if first not in string.uppercase:
errs.append(uyer-name: + first)
if buyer not in inventory.buyers( ):
msgs.append(
ew-buyer-added)
inventory.add_buyer(buyer)
validate_order(product, quantity, errs, msgs) # mutable list args
return string.join(msgs), string.join(errs) # use "(ss)" format
def validate_order(product, quantity, errs, msgs):
if product not in inventory.skus( ):
errs.append(ad-product)
elif quantity > inventory.stock(product):
errs.append(check-quantity)
else:
inventory.reduce(product, quantity)
if inventory.stock(product) / quantity < 2:
msgs.append(
eorder-soon: + `product`)
The ppembed API originally appeared as an example in the first
edition of this book. Since then, it has been utilized in real
systems and become too large to present here in its entirety. For
instance, ppembed also supports debugging embedded code (by routing
it to the pdb debugger module), dynamically
reloading modules containing embedded code, and other features too
complex to illustrate usefully here.
But if you are interested in studying another example of Python
embedding calls in action, ppembeds full source code and
makefile live in this directory on the enclosed CD (see http://examples.oreilly.com/python2):
As a sample of the kinds of tools you can build to simplify
embedding, the ppembed APIs header file is shown in Example 20-22. You are invited to study, use, copy, and
improve its code as you like. Or simply write an API of your own; the
main point to take from this section is that embedding programs need
only be complicated if you stick with the Python runtime API as
shipped. By adding convenience functions such as those in ppembed,
embedding can be as simple as you make it. It also makes your C
programs immune to changes in the Python C core; ideally, only the
API must change if Python ever does.
Be sure to also see file abstract.h in the
Python include directory if you are in the market for higher-level
interfaces. That file provides generic type operation calls that make
it easy to do things like creating, filling, indexing, slicing, and
concatenating Python objects referenced by pointer from C. Also see
the corresponding implementation file,
abstract.c, as well as the Python built-in
module and type implementations in the Python source distribution for
more examples of lower-level object access. Once you have a Python
object pointer in C, you can do all sorts of type-specific things to
Python inputs and outputs.
/*************************************************************************
* PPEMBED, VERSION 2.0
* AN ENHANCED PYTHON EMBEDDED-CALL INTERFACE
*
* Wraps Pythons run-time embedding API functions for easy use.
* Most utilities assume the call is qualified by an enclosing module
* (namespace). The module can be a file-name reference or a dummy module
* created to provide a namespace for file-less strings. These routines
* automate debugging, module (re)loading, input/output conversions, etc.
*
* Python is automatically initialized when the first API call occurs.
* Input/output conversions use the standard Python conversion format
* codes (described in the C API manual). Errors are flagged as either
* a -1 int, or a NULL pointer result. Exported names use a PP_ prefix
* to minimize clashes; names in the built-in Python API use Py prefixes
* instead (alas, there is no "import" equivalent in C, just "from*").
* Also note that the varargs code here may not be portable to certain
* C compilers; to do it portably, see the text or file vararg.txt
* here, or search for string STDARG in Pythons source code files.
*
* New in this version/edition: names now have a PP_ prefix, files
* renamed, compiles to a single .a file, fixed pdb retval bug for
* strings, and char* results returned by the "s" convert code now
* point to new char arrays which the caller should free( ) when no
* longer needed (this was a potential bug in prior version). Also
* added new API interfaces for fetching exception info after errors,
* precompiling code strings to byte code, and calling simple objects.
*
* Also fully supports Python 1.5 module package imports: module names
* in this API can take the form "package.package.[...].module", where
* Python maps the package names to a nested directories path in your
* file system hierarchy; package dirs all contain __init__.py files,
* and the leftmost one is in a directory found on PYTHONPATH. This
* APIs dynamic reload feature also works for modules in packages;
* Python stores the full path name in the sys.modules dictionary.
*
* Caveats: there is no support for advanced things like threading or
* restricted execution mode here, but such things may be added with
* extra Python API calls external to this API (see the Python/C API
* manual for C-level threading calls; see modules rexec and bastion
* in the library manual for restricted mode details). For threading,
* you may also be able to get by with C threads and distinct Python
* namespaces per Python code segments, or Python language threads
* started by Python code run from C (see the Python thread module).
*
* Note that Python can only reload Python modules, not C extensions,
* but its okay to leave the dynamic reload flag on even if you might
* access dynamically-loaded C extension modules--in 1.5.2, Python
* simply resets C extension modules to their initial attribute state
* when reloaded, but doesn actually reload the C extension file.
*************************************************************************/
#ifndef PPEMBED_H
#define PPEMBED_H
#ifdef __cplusplus
extern "C" { /* a C library, but callable from C++ */
#endif
#include
extern int PP_RELOAD; /* 1=reload py modules when attributes referenced */
extern int PP_DEBUG; /* 1=start debugger when string/function/member run */
typedef enum {
PP_EXPRESSION, /* which kind of code-string */
PP_STATEMENT /* expressions and statements differ */
} PPStringModes;
/***************************************************/
/* ppembed-modules.c: load,access module objects */
/***************************************************/
extern char *PP_Init(char *modname);
extern int PP_Make_Dummy_Module(char *modname);
extern PyObject *PP_Load_Module(char *modname);
extern PyObject *PP_Load_Attribute(char *modname, char *attrname);
extern int PP_Run_Command_Line(char *prompt);
/**********************************************************/
/* ppembed-globals.c: read,write module-level variables */
/**********************************************************/
extern int
PP_Convert_Result(PyObject *presult, char *resFormat, void *resTarget);
extern int
PP_Get_Global(char *modname, char *varname, char *resfmt, void *cresult);
extern int
PP_Set_Global(char *modname, char *varname, char *valfmt, ... /*val*/);
/***************************************************/
/* ppembed-strings.c: run strings of Python code */
/***************************************************/
extern int /* run C string of code */
PP_Run_Codestr(PPStringModes mode, /* code=expr or stmt? */
char *code, char *modname, /* codestr, modnamespace */
char *resfmt, void *cresult); /* result type, target */
extern PyObject*
PP_Debug_Codestr(PPStringModes mode, /* run string in pdb */
char *codestring, PyObject *moddict);
extern PyObject *
PP_Compile_Codestr(PPStringModes mode,
char *codestr); /* precompile to bytecode */
extern int
PP_Run_Bytecode(PyObject *codeobj, /* run a bytecode object */
char *modname,
char *resfmt, void *restarget);
extern PyObject * /* run bytecode under pdb */
PP_Debug_Bytecode(PyObject *codeobject, PyObject *moddict);
/*******************************************************/
/* ppembed-callables.c: call functions, classes, etc. */
/*******************************************************/
extern int /* mod.func(args) */
PP_Run_Function(char *modname, char *funcname, /* func|classname */
char *resfmt, void *cresult, /* result target */
char *argfmt, ... /* arg, arg... */ ); /* input arguments*/
extern PyObject*
PP_Debug_Function(PyObject *func, PyObject *args); /* call func in pdb */
extern int
PP_Run_Known_Callable(PyObject *object, /* func|class|method */
char *resfmt, void *restarget, /* skip module fetch */
char *argfmt, ... /* arg,.. */ );
/**************************************************************/
/* ppembed-attributes.c: run object methods, access members */
/**************************************************************/
extern int
PP_Run_Method(PyObject *pobject, char *method, /* uses Debug_Function */
char *resfmt, void *cresult, /* output */
char *argfmt, ... /* arg, arg... */ ); /* inputs */
extern int
PP_Get_Member(PyObject *pobject, char *attrname,
char *resfmt, void *cresult); /* output */
extern int
PP_Set_Member(PyObject *pobject, char *attrname,
char *valfmt, ... /* val, val... */ ); /* input */
/**********************************************************/
/* ppembed-errors.c: get exception data after api error */
/**********************************************************/
extern void PP_Fetch_Error_Text( ); /* fetch (and clear) exception */
extern char PP_last_error_type[]; /* exception name text */
extern char PP_last_error_info[]; /* exception data text */
extern char PP_last_error_trace[]; /* exception traceback text */
extern PyObject *PP_last_traceback; /* saved exception traceback object */
#ifdef __cplusplus
}
#endif
#endif (!PPEMBED_H)
While
writing this chapter, I ran out of space before I ran out of
examples. Besides the ppembed API example described in the last
section, you can find a handful of additional Python/C integration
self-study examples on this books CD (see http://examples.oreilly.com/python2):
The full implementation of the validation examples listed earlier.
This case study uses the ppembed API to run embedded Python order
validations, both as embedded code strings and as functions fetched
from modules. The inventory is implemented with and without shelves
and pickle files for data persistence.
A tool for exporting C variables for use in embedded Python programs. A simple ppembed test program, shown with and without package import
paths to identify modules.
Some of these are large C examples that are probably better studied
than listed.
.6.2 Running Code Strings with ppembed
Example 20-16. PP2EIntegrateEmbedApiClientscodestring-low.c
Example 20-17. PP2EIntegrateEmbedApiClientscodestring-api.c
.6.3 Running Customizable Validations
Example 20-18. PP2EIntegrateEmbedInventoryorder-string.c
Example 20-19. PP2EIntegrateEmbedInventoryvalidate1.py
Example 20-20. PP2EIntegrateEmbedInventoryorder-func.c
Example 20-21. PP2EIntegrateEmbedInventoryvalidate2.py
.6.4 ppembed Implementation
Example 20-22. PP2EIntegrateEmbedHighLevelApippembed.h
.6.5 Other Integration Examples on the CD
Категории