Saturday, November 9, 2013

Python3 Adventures

At my $dayjob, i work with python, but mostly python 2.7. As you might be aware, python 3 has been out in the wild since a very long time, and unfortunately many projects have not yet embraced it as much as python core devs would like. Our company has also been one of the many holdouts, partly because of the lack of compelling business reasons to move, not many advantages to moving (which is slowly changing with each new python 3.x minor release), and partly because there is much more higher priority work to be done.

However, thanks to some members of the community (burnpanck, who has python3 compatible (though not 100%) forks of parts of our core ETS stack, starting with traits, pyface and traitsui), this is slowly changing.
Based on his work, i have attempted to make chaco work on python3, and have largely been successful. The most significant changes were in the extension modules of enable, as expected, since the uses 2to3 to migrate python code. Enable's kiva renderer which uses AGG underneath, has significant amount of extension code. Luckily, most of it it written in swig which automatically takes care of python2 and python3 compatibility, though in one place i had to modify swig interface file for bytes<->str<->unicode madness. Chaco also has an extension module for contour plots which uses direct Python<->C api which i could make compatible with both 2 and 3 with some #ifdefs.

In short, migrating from 2 to 3 in not really difficult for most programs. 2to3 makes it even less effort, you just have to figure out the things which 2to3 cannot handle, which is easy if you have tests.

The proof:

Current status: Enable and Chaco seem to work very well under python3. All enable tests pass, though a few chaco tests fail. Also, the chaco toolbar overlay seems to segfault. All other examples in the Chaco demo run very well.
The story is not as so good for traits and traitsui though. The traits `implements` and `adapts` advisors do not work in python3. They are already deprecated in traits, so you must use the corresponding replacements from the new `traits.adaptation` package, even on python2. The same work is also required to be done for traitsui, which uses the old mechanism in quite a lot of place.

Python3 branches of ETS components:
Traits (derived from burnpanck)
Pyface (derived from burnpanck)
TraitsUI (derived from burnpanck)

PS: If you are as excited about this as I am, please help the cause by using the python3 branch in your project (they also work with python 2.7), and submit issues and pull requests for both python2 and python2 compatibility. Thanks.

PPS: If you are using a newer gcc version (tested with gcc 4.8.2) and experience segfaults in enable/chaco image plots (enable's `integration_tests/kiva/agg/` segfaults for you), then you need to disable all optimizations from the compiler (export CFLAGS='-O0'; export CXXFLAGS='-O0') while building enable because apparently gcc's optimizer eliminates some necessary code.
As an alternative, you can use the clang compiler (export CC='clang'; CXX='clang++') which doesn't exhibit this problem at any optimization level. I have yet to figure out whether it is a gcc bug of a bug in enable.