cython functions coverage revsited

A few days back i showed how to get coverage of cython functions using Ned Batchelder's coverage.py
In that post i had posted a patch to coveragepy to enable cython function coverage.

However i realize that many of my friends have never applied a patch before, dont have admin rights on few machines few other problems which may hinder their using this new feature, so here's a good new for you.

I've rewritten the patch into a single file pyx_coverage.py . You can directly use this file instead of 'coverage' command to get cython function coverage, no need to patch anything. You still need to have Ned's coveragepy installed though.

All commands/options/configuration files for coveragepy are applicable here too.

To find coverage of cython files (pyx extension) you need to do following:
1. compile cython code to 'c' with directive profile=True
2. keep source pyx files in same locations as the compiled .so files
    i.e. use 'python setup.py build_ext --inplace' or 'python setup.py develop'
3. run coverage (this file) with the option timid enabled (can also set in in .coveragerc)
    i.e. 'python pyx_coverage.py run --timid my_module.py'

You can use nose test collector as follows:
$ python pyx_coverage.py run /path/to/nosetests /path/to/source

replacing the /paths as appropriate

Download the file from here: https://sites.google.com/site/pankaj86/files/pyx_coverage.py

Reader's Bonus: If you can help me write a python c extension for this treat assured.
Hint: See Ned's coverage.tracer python c extension.

cython functions coverage using coverage.py

For all the coders out there, if you have not been writing unit tests for your code then god bless you, but if you do write tests here's another tool you must use: code coverage.
In python, on of the most popular code coverage tools is Ned Batchelder's coverage.py. It reports statement coverage of all your tests and also can report coverage in beautiful html pages.Its a very nice tool and also integrates well with testing frameworks such as nose to automate your testing and coverage reporting tasks.

But, for all those who use cython, you must surely be aware of the difficulties it brings along while testing, you can never be sure if "all is well". coverage.py doesn't report coverage of cython modules, as those are compiled into native functions.
To mitigate this problem to some extent, i wrote a simple patch to enable coverage.py to report function coverage (not statement coverage) of cython pyx files too, hurray. So now after applying the patch to coverage, all you need to do is:

1. compile cython code with profiling enabled (cython --directive profile=True)
2. run your tests under coverage as you would normally do taking care to add the timid option (coverage --timid test.py)
3. ???
4. profit

So now that you can profit, i'd be greatly thankful if someone comes up and writes this patch into the tracer.c file in the coverage source (it's very simple, get the source using $ hg clone http://bitbucket.org/ned/coveragepy)

cython timings test

The TASK : To optimize cython functions

Detailed: functions which depend on a once initialized attribute value

This often comes handy in many cases, for example to write a Laplacian function of a scalar field in spherical/axisymmetric coordinate system, you would need three independent cases for 1,2,3 dimensions for performance purposes and if u do not write all functions as general 3D functions.

The test CODE : test_kernel.pyx

cdef class Kernel:     cdef int dim     cdef double (*func)(Kernel,double)     def __init__(self, dim=1):         self.dim = dim         if dim == 1:             self.func = self.func1         elif dim == 2:             self.func = self.func2        cdef double func1(self, double x):         return 1+x        cdef double func2(self, double x):         return 2+x        cdef double c_func(self, double x):         '''this is only to make function signature compatible with func1 and func2'''         return self.func(self, x)        def p_func(self, double x):         return self.func(self, x)        cpdef double py_func(self, double x):         return self.func(self, x)        cpdef double py_c_func(self, double x):         return self.c_func(x)        def py_func1(self, x):         return self.func1(x)        def py_func2(self, x):         return self.func2(x)        cdef double func_common(self, double x):         cdef int dim = self.dim         if dim == 1:             return 10+x         elif dim == 2:             return 20+x        def py_func_c_common(self, x):         return self.func_common(x)        cpdef double py_func_common(self, double x):         cdef int dim = self.dim         if dim == 1:             return 10+x         elif dim == 2:             return 20+x

Compilation command:
    cython -a test_kernel.pyx;
    gcc <optimization-flag> -shared -fPIC test_kernel.c -lpython2.6 -I /usr/include/python2.6/ -o test_kernel.so
where optimization flag is either empty or "-O2" or "-O3"

Cython optimization
Tip 1:
Type (cdef) as many variables as you can. You also need to type the locals in each function. Try to try to use C data types wherever possible.
Tip 2:
use:
    cython -a file.pyx
command to generate a html file which shows lines which cause expensive python functions to be called. Clicking on a line shows the corresponding C code generated, highlighting expensive calls in shades of red. Try to eliminate as many such calls as you can.

The TEST :

time_kernel.py

import timeit def time(s):     '''returns time in microseconds'''     t = 1e6*timeit.timeit(s,'import test_kernel;k1=test_kernel.Kernel(1);k2=test_kernel.Kernel(2);',number=1000000)/1000000.     print s, t     return t time('k1.p_func(0)') time('k1.py_func(0)') time('k1.py_func1(0)') time('k1.py_c_func(0)') time('k1.py_func_c_common(0)') time('k1.py_func_common(0)') time('k2.p_func(0)') time('k2.py_func(0)') time('k2.py_func2(0)') time('k2.py_c_func(0)') time('k2.py_func_c_common(0)') time('k2.py_func_common(0)')

Timings :

	function	time (μs)					(ns)
	Optimization flag ->	None	-O2	-O3	sum	(k1+k2)/2	penalty
1	k1.p_func(0)	0.20178	0.18321	0.18035	0.18845	0.19368	0.0000
2	k1.py_func(0)	0.23224	0.18599	0.18393	0.20072	0.19541	1.7345
3	k1.py_func1(0)	0.21477	0.18991	0.19252	0.19907	0.19802	4.3456
4	k1.py_c_func(0)	0.23395	0.19196	0.19243	0.20611	0.19761	3.9311
5	k1.py_func_c_common(0)	0.19566	0.18458	0.19062	0.19029	0.19767	3.9960
6	k1.py_func_common(0)	0.21981	0.18707	0.18984	0.19891	0.19510	1.4237
7	k2.p_func(0)	0.20448	0.18388	0.18194	0.19010
8	k2.py_func(0)	0.21798	0.18859	0.18437	0.19698
9	k2.py_func2(0)	0.20413	0.18124	0.18194	0.18910
10	k2.py_c_func(0)	0.23114	0.19166	0.19238	0.20506
11	k2.py_func_c_common(0)	0.19860	0.18783	0.18745	0.19129
12	k2.py_func_common(0)	0.21609	0.18747	0.18640	0.19666
	Average	0.21560	0.18681	0.18703	0.19648

Result :

The best is to write separate C function and a python accessor function.

task	function	penalty cost (ns)
C function + python accessor : base case	p_func
cpdef instead of def	py_func	1.7345
calling a cdef class method instead of a function pointer attribute	py_func1,py_func2	4.3456
one extra c function call	py_c_func	3.9311
(def + cdef) instead of (cpdef)	py_func_c_common-py_func_common	2.5723
One C comparison vs one C function call	py_func_common	1.4237

Conclusion :

As can be clearly seen that the results are clearly inconclusive :)
This was a small test carried on my laptop with no controlled environment. Also thought the results seemed close to repeatable, nevertheless many trials should be conduction and each value should have a standard deviation also to check the repeatability. However one clear conclusion is do not forget to add optimization flags. Setuptools already does that for you.
Also using a function pointer is not so bad after all. It would become more advantageous in case of more number of comparisons.
Cython provides great speedups (who didn't know that :) ). The pure python version of py_func_common took 0.408μs for dim=1 and 0.518μs for dim=2
These results are purely from python point of view. The effect of cdef/cpdef should also be considered in c/cython code which calls these functions.

CAVEAT:

I am no optimization expert. I have done this out of out of sheer boredom :)
If anyone wants to verify, you are welcome
Any information content is purely coincindental

Easier cython/python/c debugging with new GDB

We all know how debugging is an dreaded integral part of every programmer's work. It can also be fun sometimes depending on the time to deadline, complexity of the bug and time already spent.
So if anyone is still left who does assignments or other programs without debugging (using print statements etc) then please consider learning it. Else you are simply increasing your work and frustration.
For debugging in any programming language my advise would be to use the eclipse debugger gui which provides all the standard features present in any debugger and integration with java, c/c++, python and a host of other languages.

This post was not about plain debugging. Its about the new features in GDB 7 (the GNU debugger) which enables writing pretty printers in python. More information can be had from the net. However it means a much easier debugging experience with cython. The good folks at Fedora have written some cool scripts to integration python scripting capability of gdb to enable easier cython debugging.
Check out the awesomeness at https://fedoraproject.org/wiki/Features/EasierPythonDebugging
In short now you can do the following easily with the new gdb

• automatically display python frame information in PyEval_EvalFrameEx in gdb backtraces, including in ABRT:
• python source file, line number, and function names
• values of locals, if available
• name of function for wrapped C functions

 This gonna make my life easier, especially since my DD project is in cyton/python.

Also not to forget the uber cool features it could enable not only for python developers but for all. As an example check out the blog at http://labs.trolltech.com/blogs/2010/04/22/peek-and-poke-vol-3/