Compiling PyXMLSec for Windows

For some reason there doesn't seem to be anyone that compiled PyXMLSec for Windows. Not sure why but I tried searching for all sorts of terms and nada. So I went on a bit of an adventure to get this thing working. Fair warning: while this works for the most part I mixed and matched a few of the binary packages without really researching the platform they were compiled on and whether they really would be compatible.

If you are looking to just download the binary go here:
http://returnbooleantrue.blogspot.com/2009/04/pyxmlsec-windows-binary.html


List of everything I used:

- MinGW. I used version 5.1.4.
- PyXMLSec source code. For this I used pyxmlsec-0.3.0.
- libxml2-2.5.10.win32
- iconv-1.9.2.win32
- libxmlsec-1.2.11.win32
- libxslt-1.1.24.win32
- zlib-1.2.3.win32
- Win32OpenSSL-0_9_8i.exe and Microsoft Visual C++ 2008 Redistributable Package (x86) from the same site



First thing I did was look at the setup.py script included inside pyxmlsec.

A bit down in the file is a list of all the files that need to be compiled.

sources = ["utils.c", "wrap_objs.c",
"app.c", "base64.c", "buffer.c", "errors.c",
"keyinfo.c", "keys.c", "keysdata.c", "keysmngr.c",
"list.c", "membuf.c", "nodeset.c", "parser.c",
"templates.c", "transforms.c", "version.c",
"xmldsig.c", "xmlenc.c", "xmlsec.c", "xmltree.c",
"x509.c",
"xmlsecmod.c"]



Looked pretty straghtforward to me. Took out my compiler and proceded to start:

> gcc *.c

One can hope right? Obviously all this does is spew out a ridiculous number of errors and warnings. So I set about troubleshooting them one by one. I decide to just pick a single .c file to just get a small number of warnings. I was too lazy to pipe output to a file.

> gcc utils.c

In file included from utils.c:27:
utils.h:5:20: Python.h: No such file or directory

well this is simple enough to fix. utils.c is expecting the Python.h header file. So lets give the compiling the path to the headers using the -I option.

> gcc -IC:\Python25\include utils.c

utils.c:(.text+0x4f): undefined reference to `_imp__PyInstance_Type'

So these turn out to be linking errors. So let us link to the Python libraries. We need to include the -lpython25 to explicitly tell gcc that the library we want is python25.

> gcc -IC:\Python25\include -LC:\Python25\libs utils.c -lpython25

utils.c:(.text+0x8e): undefined reference to `xmlsec_error'

So now I get errors about linking to the xmlsec library. Well this makes sense since I didn't tell gcc where the xmlsec libraries were. PyXMLSec depends on xmlsec.

At this point I am dreading I will have to compile xmlsec for windows as well. Luckily windows binaries for the XMLSec Library (as well as LibXML2, LibXSLT and OpenSSL) are available from Igor Zlatkovic. I will make a big caveat here though. Some of the binaries that I got from Igor did not work for me. One of the errors was about newline characters being missing in his openssl binary package. To solve this I downloaded

Win32 OpenSSL v0.9.8i from Shining Light Productions (http://www.slproweb.com/products/Win32OpenSSL.html) and linked against that.

The other issue was with libxml2. Don't try to use the newest version or else you will run into issues.

Now here is the relatively time consuming part. Need to unpack them all and make sure gcc knows about the locations.

This is what we have so far.
> gcc -IC:\Python25\include -I -LC:\Python25\libs utils.c -lpython25

Add in all the libraries and we get:

gcc -IC:\Python25\include -IC:\Build\iconv-1.9.2.win32\include-IC:\Build\libxml2-2.7.2.win32\include

-IC:\Build\libxmlsec-1.2.11.win32\include
-IC:\Build\libxslt-1.1.24.win32\include
-IC:\Build\openssl-0.9.8a.win32\include
-IC:\Build\zlib-1.2.3.win32\include -LC:\Python25\libs
-LC:\Build\iconv-1.9.2.win32\lib -LC:\Build\libxml2-2.7.2.win32\lib
-LC:\Build\libxmlsec-1.2.11.win32\lib
-LC:\Build\libxslt-1.1.24.win32\lib
-LC:\Build\openssl-0.9.8a.win32\lib -LC:\Build\zlib-1.2.3.win32\lib
parser.c -lpython25 -llibxml2 -llibxmlsec -llibxslt -libexslt
-llibxmlsec-openssl -lbz2 -liconv -lzlib

Looks really ugly but running it gives us a glimmer of hope:

In file included from xmlsecmod.h:4,
from parser.c:25:
C:/Build/libxmlsec-1.2.11.win32/include/xmlsec/crypto.h:54:2: #error
No crypto library defined

So we need to pick a crypto library. We do this through the gcc option -D.

-DXMLSEC_CRYPTO_OPENSSL

While looking for define options I also happen upon a post that said
to define -DLIBXML_STATIC -DLIBXSLT_STATIC -DXMLSEC_STATIC.

Another try:

gcc -DXMLSEC_CRYPTO_OPENSSL -DLIBXML_STATIC -DLIBXSLT_STATIC
-DXMLSEC_STATIC -IC:\Python25\include
-IC:\Build\iconv-1.9.2.win32\include
-IC:\Build\libxml2-2.5.10.win32\include
-IC:\Build\libxmlsec-1.2.11.win32\include
-IC:\Build\libxslt-1.1.24.win32\include -IC:\OpenSSL\include
-IC:\Build\zlib-1.2.3.win32\include -LC:\Python25\libs
-LC:\Build\iconv-1.9.2.win32\lib -LC:\Build\libxml2-2.5.10.win32\lib
-LC:\Build\libxmlsec-1.2.11.win32\lib
-LC:\Build\libxslt-1.1.24.win32\lib -LC:\OpenSSL\lib\MinGW
-LC:\Build\zlib-1.2.3.win32\lib *.c -lpython25 -llibxml2 -llibxmlsec
-llibxslt -llibexslt -llibxmlsec-openssl -lbz2 -liconv -lzlib


base64.c:(.text+0x5a1): undefined reference to `xmlMalloc'
base64.c:(.text+0x605): undefined reference to `xmlFree'

This is what you'd get if you tried to use a newer libxml2. I tried these versions:

libxml2-2.7.2.win32 - Doesn't work.
libxml2-2.6.9.win32 - Doesn't work.
libxml2-2.5.10.win32 - Works.

The last error I got was:
/mingw/lib/libmingw32.a(main.o):main.c:(.text+0xbd): undefined
reference to `WinMain@16'

To fix that I added a -shared option to gcc. I also added a -o option
so I wouldn't just get a a.exe.

gcc -DXMLSEC_CRYPTO_OPENSSL -DLIBXML_STATIC -DLIBXSLT_STATIC
-DXMLSEC_STATIC -IC:\Python25\include
-IC:\Build\iconv-1.9.2.win32\include
-IC:\Build\libxml2-2.5.10.win32\include
-IC:\Build\libxmlsec-1.2.11.win32\include
-IC:\Build\libxslt-1.1.24.win32\include -IC:\OpenSSL\include
-IC:\Build\zlib-1.2.3.win32\include -LC:\Python25\libs
-LC:\Build\iconv-1.9.2.win32\lib -LC:\Build\libxml2-2.5.10.win32\lib
-LC:\Build\libxmlsec-1.2.11.win32\lib
-LC:\Build\libxslt-1.1.24.win32\lib -LC:\OpenSSL\lib\MinGW
-LC:\Build\zlib-1.2.3.win32\lib *.c -lpython25 -llibxml2 -llibxmlsec
-llibxslt -llibexslt -llibxmlsec-openssl -lbz2 -liconv -lzlib -shared
-o pyxmlsec.pyd

Now that I know everything compiles cleanly I decide to go back and try my luck editing setup.py. To be honest I just deleted a lot of stuff and added what I needed.


from distutils.core import setup, Extension
import sys, commands


define_macros = []
include_dirs = []
library_dirs = []
libraries = []

define_macros.append(("XMSEC_CRYPTO_OPENSSL",None))
define_macros.append(("LIBXML_STATIC",None))
define_macros.append(("LIBXSLT_STATIC",None))
define_macros.append(("XMLSEC_STATIC",None))

include_dirs.append("C:\\Build\\iconv-1.9.2.win32\\include")
include_dirs.append("C:\\Build\\libxml2-2.5.10.win32\\include")
include_dirs.append("C:\\Build\\libxmlsec-1.2.11.win32\\include")
include_dirs.append("C:\\Build\\libxslt-1.1.24.win32\\include")
include_dirs.append("C:\\OpenSSL\\include")
include_dirs.append("C:\\Build\\zlib-1.2.3.win32\\include")

library_dirs.append("C:\\Build\\iconv-1.9.2.win32\\lib")
library_dirs.append("C:\\Build\\libxml2-2.5.10.win32\\lib")
library_dirs.append("C:\\Build\\libxmlsec-1.2.11.win32\\lib")
library_dirs.append("C:\\Build\\libxslt-1.1.24.win32\\lib")
library_dirs.append("C:\\OpenSSL\\lib\\MinGW")
library_dirs.append("C:\\Build\\zlib-1.2.3.win32\\lib")

libraries.append("python25")
libraries.append("libxml2")
libraries.append("libxmlsec")
libraries.append("libxslt")
libraries.append("libexslt")
libraries.append("libxmlsec-openssl")
libraries.append("bz2")
libraries.append("iconv")
libraries.append("zlib")

em = Extension("xmlsecmod",
sources = ["utils.c", "wrap_objs.c",
"app.c", "base64.c", "buffer.c", "errors.c",
"keyinfo.c", "keys.c", "keysdata.c", "keysmngr.c",
"list.c", "membuf.c", "nodeset.c", "parser.c",
"templates.c", "transforms.c", "version.c",
"xmldsig.c", "xmlenc.c", "xmlsec.c", "xmltree.c",
"x509.c",
"xmlsecmod.c"],
define_macros = define_macros,
include_dirs = include_dirs,
library_dirs = library_dirs,
libraries = libraries
)

doclines = __doc__.split("\n")

setup(name = "pyxmlsec",
version = "0.3.0",
description = doclines[0],
long_description = "\n" . join(doclines[2:]),
author = "Valery Febvre",
author_email = "vfebvre@easter-eggs.com",
license = "GNU GPL",
platforms = ["any"],
url = "http://pyxmlsec.labs.libre-entreprise.org",
ext_modules = [em],
py_modules = ["xmlsec", "xmlsec_strings"]
)

Ran setup.py with the following command:

C:\Python25\python.exe setup.py build --compiler=mingw32 install

running build
running build_py
running build_ext
running install
running install_lib
copying build\lib.win32-2.5\xmlsec.py -> C:\Python25\Lib\site-packages
copying build\lib.win32-2.5\xmlsecmod.pyd -> C:\Python25\Lib\site-packages
copying build\lib.win32-2.5\xmlsec_strings.py -> C:\Python25\Lib\site-packages
byte-compiling C:\Python25\Lib\site-packages\xmlsec.py to xmlsec.pyc
byte-compiling C:\Python25\Lib\site-packages\xmlsec_strings.py to
xmlsec_strings.pyc
running install_egg_info
Writing C:\Python25\Lib\site-packages\pyxmlsec-0.3.0-py2.5.egg-info

Everything looks good. Startup python and try to import xmlsec. At this point you'll probably get a few errors about missing DLLs and such.

I had to install Libxml and Libxslt Python Bindings for Windows
http://users.skynet.be/sbi/libxml-python/


>>> import xmlsec
Traceback (most recent call last):
File "", line 1, in
File "xmlsec.py", line 46, in
import xmlsecmod
ImportError: DLL load failed: The specified module could not be found.

A few other things I had to do:

Copied C:\Build\libxmlsec-1.2.11.win32\bin\libxmlsec-openssl.dll to C:\Python25
Copied C:\Build\libxmlsec-1.2.11.win32\bin\libxmlsec.dll to C:\Python25
Copied C:\Build\libxmlsec-1.2.11.win32\bin\libxslt.dll to C:\Python25
Copied C:\Build\libxmlsec-1.2.11.win32\bin\libexslt.dll to C:\Python25
Copied C:\Build\libxmlsec-1.2.11.win32\bin\libxmlsec-openssl.dll to C:\Python25

I also downloaded a Windows dependency walker to troubleshoot some OpenSSL DLL issues. The solution was to reinstall OpenSSL and have it put its DLLs in my Windows system32 directory.

C:\Python25>python
Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import xmlsec
>>>

In the end I was rewarded with a nice clean import of xmlsec. The examples even work!