Prev: how to judge urllib.Request is finished?
Next: Having problems using Tkinter
From: Marcin Krol on 1 Jul 2008 05:36
Hello everyone,

I'm trying to embed Python interpreter in C code, but in a specific way:
loading compiled bytecode into a memory location and executing it (don't
ask why, complicated reasons).

PyImport_ExecCodeModule seems like obvious candidate, docs say:

"Given a module name (possibly of the form package.module) and a code
object read from a Python bytecode file or obtained from the built-in
function compile(), load the module."

Code:

---cut---
#include <Python.h>
#include <stdio.h>
#include <stdlib.h>
#include <syslog.h>
#include <unistd.h>


int load_file(char *fname, unsigned char** result)
{
int size = 0;
FILE *f = fopen(fname, "rb");
if (f == NULL)
{
*result = NULL;
return -1;
}
fseek(f, 0, SEEK_END);
size = ftell(f);
*result = (unsigned char *) malloc(size+1);
fseek(f, 0, SEEK_SET);
size = fread(*result, sizeof(unsigned char), size, f);
return size;
}

int main(int argc, char **argv)
{
int size;
unsigned char *python_code;
PyObject *mainobj;
size = load_file("multiply.pyc", &python_code);

Py_Initialize();
mainobj = PyImport_ExecCodeModule("multiply", (PyObject *)
python_code);
Py_Finalize();

}
---cut---

Compiling it following way works fine:

${CC} testit.c -g -o testit -I/usr/include/python2.4 -lpython2.4 -lm
-lutil -lpthread -ldl -L/usr/lib/python2.4/config


However, the damn thing crashes on this call:

33 mainobj = PyImport_ExecCodeModule("multiply", (PyObject
*) python_code);
(gdb) n

Program received signal SIGSEGV, Segmentation fault.
0x0804e7f6 in PyImport_ExecCodeModuleEx ()

The .pyc file woks just fine in Python interpreter:

>>> import multiply
>>> multiply.multiply()
The result of 12345 x 6789 : 83810205
83810205
>>>


What I am doing wrong? Please help.




From: Carsten Haese on 1 Jul 2008 09:41
Marcin Krol wrote:
> Hello everyone,
>
> I'm trying to embed Python interpreter in C code, but in a specific way:
> loading compiled bytecode into a memory location and executing it (don't
> ask why, complicated reasons).
>
> PyImport_ExecCodeModule seems like obvious candidate, docs say:
>
> "Given a module name (possibly of the form package.module) and a code
> object read from a Python bytecode file or obtained from the built-in
> function compile(), load the module."
> [...]
> mainobj = PyImport_ExecCodeModule("multiply", (PyObject *)
> python_code);
> [...]

python_code is a C string containing the raw bytes from your pyc file.
Casting that to a PyObject pointer will not magically transform it into
a Python code object. A pyc file contains the following:

1) An 8 byte header containing a magic number.
2) A "marshal" serialization of the code object.

So, in order to transform those contents into a code object, you need to
skip the 8 byte header and an unmarshal the rest. Basically, replace the
line above with something like this:

codeobj = PyMarshal_ReadObjectFromString(python_code+8, size-8);
mainobj = PyImport_ExecCodeModule("multiply", codeobj);

where codeobj is of type (PyObject *).

Once that works, add magic number checking and exception handling to taste.

Hope this helps,

--
Carsten Haese
http://informixdb.sourceforge.net
From: Marcin Krol on 1 Jul 2008 12:06
Pau Freixes wrote:
> If you search this function name into Google you will can found some
> examples, use for example PyMarshal_ReadObjectFromString or
> PyMarshal_ReadObjectFromFile

Rest assured I looked at a lot of code, but none of it was reading
in the .pyc file. I had no idea about the 8-byte header in the .pyc
file, Carsten was so kind to show it.

From: Marcin Krol on 1 Jul 2008 12:07
Carsten Haese wrote:

> python_code is a C string containing the raw bytes from your pyc file.
> Casting that to a PyObject pointer will not magically transform it into
> a Python code object.

<scratching my head> well yeah, I kind of didn't think that through..


A pyc file contains the following:
>
> 1) An 8 byte header containing a magic number.
> 2) A "marshal" serialization of the code object.
>
> So, in order to transform those contents into a code object, you need to
> skip the 8 byte header and an unmarshal the rest. Basically, replace the
> line above with something like this:
>
> codeobj = PyMarshal_ReadObjectFromString(python_code+8, size-8);
> mainobj = PyImport_ExecCodeModule("multiply", codeobj);
>
> where codeobj is of type (PyObject *).
>
> Once that works, add magic number checking and exception handling to taste.


Thanks. That's exactly what I was looking for.

From: Marcin Krol on 1 Jul 2008 12:11
Hello everyone,

In the meantime I managed to work out another solution, mainly thanks to
reading the source code of some OSS projects. I post it here so somebody
else looking for solution to this problem had the example available:

----cut----
#include <Python.h>
#include <stdio.h>
#include <stdlib.h>
#include <syslog.h>
#include <unistd.h>

static FILE *read_module(const char *cpathname, long mtime)
{
FILE *fp;
long magic;
long pyc_mtime;

fp = fopen (cpathname, "rb");
if (fp == NULL)
return NULL;

magic = PyMarshal_ReadLongFromFile (fp);
if (magic != PyImport_GetMagicNumber())
{
fclose(fp);
return NULL;
}

pyc_mtime = PyMarshal_ReadLongFromFile (fp);
if (mtime && pyc_mtime != mtime)
{
fclose(fp);
return NULL;
}

return fp;
}


int main(int argc, char **argv)
{
int size;

Py_Initialize();
PyCodeObject *mainobj, *pycode;
PyObject *result, *mainmodule, *maindict;
mainmodule = PyImport_AddModule("__main__");
maindict = PyModule_GetDict(mainmodule);

FILE *f = read_module("multiply.pyc", 0L);
pycode = (PyCodeObject *) PyMarshal_ReadObjectFromFile(f);
printf("pointer: %d\n", pycode);

PyEval_EvalCode(pycode, maindict, maindict);
Py_Finalize();

}

----cut----

 | 
Pages: 1
Prev: how to judge urllib.Request is finished?
Next: Having problems using Tkinter