Programming Python (199 page)
As you can
probably tell from the preceding overview, there is much
flexibility in the embedding domain. To illustrate common embedding
techniques in action, this section presents a handful of short C programs
that run Python code in one form or another. Most of these examples will
make use of the simple Python module file shown in
Example 20-22
.
Example 20-22. PP4E\Integrate\Embed\Basics\usermod.py
"""
#############################################################
C code runs Python code in this module in embedded mode.
Such a file can be changed without changing the C layer.
This is just standard Python code (C handles conversions).
Must be on the Python module search path if imported by C.
C can also run code in standard library modules like string.
#############################################################
"""
message = 'The meaning of life...'
def transform(input):
input = input.replace('life', 'Python')
return input.upper()
If you know any Python at all, you probably know that this file
defines a string and a function; the function returns whatever it is
passed with string substitution and uppercase conversions applied. It’s
easy to use from Python:
.../PP4E/Integrate/Embed/Basics$python>>>import usermod# import a module
>>>usermod.message# fetch a string
'The meaning of life...'
>>>usermod.transform(usermod.message)# call a function
'THE MEANING OF PYTHON...'
With a little Python API wizardry, it’s not much more difficult to
use this module the same way in C.
Perhaps the simplest
way to run Python code from C is by calling thePyRun_SimpleStringAPI function. With it, C
programs can execute Python programs represented as C character string
arrays. This call is also very limited: all code runs in the same
namespace (the module__main__), the
code strings must be Python statements (not expressions), and there is
no direct way to communicate inputs or outputs with the Python code
run.
Still, it’s a simple place to start. Moreover, when augmented with
an imported C extension module that the embedded Python code can use to
communicate with the enclosing C layer, this technique can satisfy many
embedding goals. To demonstrate the basics, the C program in
Example 20-23
runs Python code to
accomplish the same results as the Python interactive session listed in
the prior section.
Example 20-23. PP4E\Integrate\Embed\Basics\embed-simple.c
/*******************************************************
* simple code strings: C acts like the interactive
* prompt, code runs in __main__, no output sent to C;
*******************************************************/
#include/* standard API def */
main() {
printf("embed-simple\n");
Py_Initialize();
PyRun_SimpleString("import usermod"); /* load .py file */
PyRun_SimpleString("print(usermod.message)"); /* on Python path */
PyRun_SimpleString("x = usermod.message"); /* compile and run */
PyRun_SimpleString("print(usermod.transform(x))");
Py_Finalize();
}
The first thing you should notice here is that when Python is
embedded, C programs always callPy_Initializeto initialize linked-in Python
libraries before using any other API functions and normally callPy_Finalizeto shut the interpreter
down.
The rest of this code is straightforward—C submits hardcoded
strings to Python that are roughly what we typed interactively. In fact,
we could concatenate all the Python code strings here with\ncharacters between, and submit it once as a
single string. Internally,PyRun_SimpleStringinvokes
the Python compiler and interpreter to run the strings
sent from C; as usual, the Python compiler is always available in
systems that contain Python.
To build a standalone executable from this C source file, you
need to link its compiled form with the Python library file. In this
chapter, “library” usually means the binary library file that is
generated when Python is compiled, not the Python source code standard
library.
Today, everything in Python that you need in C is compiled into
a single Python library file when the interpreter is built (e.g.,
libpython3.1.dll
on Cygwin). The program’smainfunction comes from your C
code, and depending on your platform and the extensions installed in
your Python, you may also need to link any external libraries
referenced by the Python library.
Assuming no extra extension libraries are needed,
Example 20-24
is a minimal makefile
for building the C program in
Example 20-23
under
Cygwin on Windows. Again, makefile details vary per
platform, but see Python manuals for hints. This makefile uses the
Python include-files path to find
Python.h
in the
compile step and adds the Python library file to the final link step
to make API calls available to the C program.
Example 20-24. PP4E\Integrate\Embed\Basics\makefile.1
# a Cygwin makefile that builds a C executable that embeds
# Python, assuming no external module libs must be linked in;
# uses Python header files, links in the Python lib file;
# both may be in other dirs (e.g., /usr) in your install;
PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
embed-simple: embed-simple.o
gcc embed-simple.o -L$(PYLIB) -lpython3.1 -g -o embed-simple
embed-simple.o: embed-simple.c
gcc embed-simple.c -c -g -I$(PYINC)
To build a program with this file, launchmakeon it as usual (as before, make sure
indentation in rules is tabs in your copy of this makefile):
.../PP4E/Integrate/Embed/Basics$make -f makefile.1gcc embed-simple.c -c -g -I/usr/local/include/python3.1
gcc embed-simple.o -L/usr/local/bin -lpython3.1 -g -o embed-simple
Things may not be quite this simple in practice, though, at
least not without some coaxing. The makefile in
Example 20-25
is the one I actually
used to build all of this section’s C programs on Cygwin.
Example 20-25. PP4E\Integrate\Embed\Basics\makefile.basics
# cygwin makefile to build all 5 basic embedding examples at once
PYLIB = /usr/local/bin
PYINC = /usr/local/include/python3.1
BASICS = embed-simple.exe \
embed-string.exe \
embed-object.exe \
embed-dict.exe \
embed-bytecode.exe
all: $(BASICS)
embed%.exe: embed%.o
gcc embed$*.o -L$(PYLIB) -lpython3.1 -g -o $@
embed%.o: embed%.c
gcc embed$*.c -c -g -I$(PYINC)
clean:
rm -f *.o *.pyc $(BASICS) core
On some platforms, you may need to also link in other libraries
because the Python library file used may have been built with external
dependencies enabled and required. In fact, you may have to link in
arbitrarily many more externals for your Python library, and frankly,
chasing down all the linker dependencies can be tedious. Required
libraries may vary per platform and Python install, so there isn’t a
lot of advice I can offer to make this process simple (this is C,
after all). The standard C development techniques will apply.
One hint here: if you’re going to do much embedding work and you
run into external dependency issues, on some platforms you might want
to build Python on your machine from its source with all unnecessary
extensions
disabled
in its build configuration
files (see the Python source package for details). This produces a
Python library with minimal external requirements, which may links
more easily.
Once you’ve gotten the makefile to work, run it to build all of
this section’s C programs at once with Python libraries linked
in:
.../PP4E/Integrate/Embed/Basics$make -f makefile.basics cleanrm -f *.o *.pyc embed-simple.exe embed-string.exe embed-object.exe
embed-dict.exe embed-bytecode.exe core
.../PP4E/Integrate/Embed/Basics$make -f makefile.basicsgcc embed-simple.c -c -g -I/usr/local/include/python3.1
gcc embed-simple.o -L/usr/local/bin -lpython3.1 -g -o embed-simple.exe
gcc embed-string.c -c -g -I/usr/local/include/python3.1
gcc embed-string.o -L/usr/local/bin -lpython3.1 -g -o embed-string.exe
gcc embed-object.c -c -g -I/usr/local/include/python3.1
gcc embed-object.o -L/usr/local/bin -lpython3.1 -g -o embed-object.exe
gcc embed-dict.c -c -g -I/usr/local/include/python3.1
gcc embed-dict.o -L/usr/local/bin -lpython3.1 -g -o embed-dict.exe
gcc embed-bytecode.c -c -g -I/usr/local/include/python3.1
gcc embed-bytecode.o -L/usr/local/bin -lpython3.1 -g -o embed-bytecode.exe
rm embed-dict.o embed-object.o embed-simple.o embed-bytecode.o embed-string.o
After building with either makefile, you can run the resulting C
program as usual:
.../PP4E/Integrate/Embed/Basics$./embed-simpleembed-simple
The meaning of life...
THE MEANING OF PYTHON...
Most of this output is produced by Pythonprintstatements sent from C to the
linked-in Python library. It’s as if C has become an interactive
Python programmer.
Naturally, strings of Python code run by C probably would not be
hardcoded in a C program file like this. They might instead be loaded
from a text file or GUI, extracted from HTML or XML files, fetched
from a persistent database or socket, and so on. With such external
sources, the Python code strings that are run from C could be changed
arbitrarily without having to recompile the C program that runs them.
They may even be changed on site, and by end users of a system. To
make the most of code strings, though, we need to move on to more
flexible API
tools.
Pragmatic details
: Under Python 3.1 and
Cygwin on Windows, I had to first set myPYTHONPATHto include the current
directory in order to run the embedding examples, with the shell
commandexport PYTHONPATH=.. I
also had to use the shell command./embed-simpleto execute the program
because.was also not on my
systempathsetting and isn’t
initially when you install Cygwin.
Your mileage may vary; but if you have trouble, try running
the embedded Python commandsimportand
sysprint sys.pathfrom C to see what Python’s path looks like, and take a look at the
Python/C API
manual for more on path
configuration for embedded applications.
Example 20-26
uses the
following API calls to run code strings that return
expression results back to C:
Py_InitializeInitializes
linked-in Python libraries as beforePyImport_ImportModuleImports a Python
module and returns a pointer to itPyModule_GetDictFetches a
module’s attribute dictionary objectPyRun_StringRuns a
string of code in explicit namespacesPyObject_SetAttrStringAssigns
an object attribute bynamestringPyArg_ParseConverts a
Python return value object to C form
The import calls are used to fetch the namespace of theusermodmodule listed in
Example 20-22
so that code strings
can be run there directly and will have access to names defined in that
module without qualifications.Py_Import_ImportModuleis like a Pythonimportstatement, but the imported
module object is returned to C; it is not assigned to a Python variable
name. As a result, it’s probably more similar to the Python__import__built-in function.
ThePyRun_Stringcall is the
one that actually runs code here, though. It takes a code string, a
parser mode flag, and dictionary object pointers to serve as the global
and local namespaces for running the code string. The mode flag can bePy_eval_inputto run an expression orPy_file_inputto run a statement;
when running an expression, the result of evaluating the expression is
returned from this call (it comes back as aPyObject*object pointer). The two namespace
dictionary pointer arguments allow you to distinguish global and local
scopes, but they are typically passed the same dictionary such that code
runs in a single namespace.
Example 20-26. PP4E\Integrate\Embed\Basics\embed-string.c
/* code-strings with results and namespaces */
#include
main() {
char *cstr;
PyObject *pstr, *pmod, *pdict;
printf("embed-string\n");
Py_Initialize();
/* get usermod.message */
pmod = PyImport_ImportModule("usermod");
pdict = PyModule_GetDict(pmod);
pstr = PyRun_String("message", Py_eval_input, pdict, pdict);
/* convert to C */
PyArg_Parse(pstr, "s", &cstr);
printf("%s\n", cstr);
/* assign usermod.X */
PyObject_SetAttrString(pmod, "X", pstr);
/* print usermod.transform(X) */
(void) PyRun_String("print(transform(X))", Py_file_input, pdict, pdict);
Py_DECREF(pmod);
Py_DECREF(pstr);
Py_Finalize();
}
When compiled and run, this file produces the same result as its
predecessor:
.../PP4E/Integrate/Embed/Basics$./embed-stringembed-string
The meaning of life...
THE MEANING OF PYTHON...
However, very different work goes into producing this output. This
time, C fetches, converts, and prints the value of the Python module’smessageattribute directly by running
a string expression and assigning a global variable (X) within the module’s namespace to serve as
input for a Pythonprintstatement
string.
Because the string execution call in this version lets you specify
namespaces, you can better partition the embedded code your system
runs—each grouping can have a distinct namespace to avoid overwriting
other groups’ variables. And because this call returns a result, you can
better communicate with the embedded code; expression results are
outputs, and assignments to globals in the namespace in which code runs
can serve as inputs.
Before we move on, I need to explain three coding issues here.
First, this program also decrements the reference count on objects
passed to it from Python, using thePy_DECREFcall described in Python’s C API
manuals. These calls are not strictly needed here (the objects’ space is
reclaimed when the programs exits anyhow), but they demonstrate how
embedding interfaces must manage reference counts when Python passes
object ownership to C. If this was a function called from a larger
system, for instance, you would generally want to decrement the count to
allow Python to reclaim the
objects
.
Second, in a realistic program, you should generally test the
return values of
all
the API calls in this program
immediately to detect errors (e.g., import failure). Error tests are
omitted in this section’s example to keep the code simple, but they
should be included in your programs to make them more robust.
And third, there is a related function that lets you run entire
files
of code, but it is not demonstrated in this
chapter:PyRun_File. Because you can
always load a file’s text and run it as a single code string withPyRun_String, thePyRun_Filecall’s main advantage is to avoid
allocating memory for file content. In such multiline code strings, the\ncharacter terminates lines and
indentation group blocks as
usual.