Plugins’ Headache

As I was writing a plugin, or to be more accurate, an extension module for Python in C/Win32; I had mysterious crashings here and there. Using a JIT I saw that the crash is in malloc/free so I looked at the stack and saw where’s the problem at (function scope). So far so good, piece of cake. Isolating the problematic code, yet I still didn’t fully understand why it happens, I ran it on a new clean project in Debug mode and got the cool message that there might be a heap corruption. Not surprising, of course. Now, the thing that made me really angry was that my extension used to work with an older version of MSVS. So thinking of this fact, I immediately took a look at python25.dll and saw that it uses MSVCRT version 7. Also, looking at my recently compiled extension dll, I noticed that my CRT is version 8. Knowing all this: heap does problems and different CRTs, I understood that my code uses one heap and Python’s uses another heap. Which is a big no-no, of course, mixing between different heaps…ouch.

Well, it was time to find a solution to a simple problem – I have to use the same heap. But how? Well, to make a short story shorter, I took a look at the Python’s header files and noticed they have two malloc’s. One was a macro which substitutes with malloc(size), which means use the current compiler’s CRT (that what I had used actually). And the other malloc was their own implementation of memory-allocation (above their own CRT’s malloc). Reading the comments surrounding that function declaration it says something like “platform independent allocations”. So voila, I knew that all I had to do is using these memory functions. At the moment I saw these functions, I blessed Python’s developers for thinking, whether directly or not, about this problem :)

The moral story is that if you integrate your code with another application in the code level make sure you talk the same protocol. It might be a memory heap, in my case, and it might be different implementation to FILE*… Just remember to supply the necessary functions for different compilers. Or if you’re on the SDK users side, use the SDK wisely, before you start hacking your way to a working solution.

Leave a Reply