Notes on getting TileStache renders working on MacOS

Personal work notes for this project I’m working on. Mac DLL hell included. Unlikely to be of interest to any unfortunate reader who found this post.

I’m trying to run some tile generation software that generates odd TIFFs. The TIFFs have two separate images in one single file. This is a legitimate TIFF but it’s unusual and I’m having a hard time producing it on my Mac. The exact same end code works fine on Linux.

There’s a bunch of software involved here. Python, gdal, boost, MapNik, Cairo, swig, TileStache, … When it’s all mashed up the code I’m running is basically a Python program with a lot of bindings to geo and image C libraries, using boost and swig. Ie: a big fucking hairball.

If I use MacOS Lion’s version of libtiff, the code all runs fine but the images it produces are corrupted. Same error as this discussion. gdalinfo can’t load them:

gdalinfo out/10/000/163/000/395.tiff
ERROR 1: out/10/000/163/000/395.tiff:Failed to allocate memory for to read TIFF directory (0 elements of 12 bytes each)
ERROR 1: TIFFReadDirectory:Failed to read directory at offset 52744
gdalinfo failed - unable to open 'out/10/000/163/000/395.tiff'.

If I use homebrew’s default libtiff (3.9.5) the code hangs. Poking at it with gdb I see the program is spinning at 100% CPU deep inside some error binding between libtiff, Python, and either Boost or Swig. Yuck!

If I build my own tiff-4.0.0beta7 in homebrew and use it, it runs and produces the same corrupted TIFFs as Lion. (To build in Homebrew, you have to edit the libtiff.rb formula and add an option to configure, “--x-includes=/usr/X11/include”. Otherwise, it can’t find some of the GLheader files when compiling.). tiff-4.0.0beta5 is no better.

A big confounding issue has been getting the Python environment right in Homebrew. More notes on that here. I think everything’s working anyway, various Python and gdal programs do, but you never know.

Another confounding issue has been compiling things correctly. The transition in C compilers on Lion makes a mess. Homebrew gamely tries to use LLVM but some stuff only works with gcc-4.2 (including Numpy). I was worried that mixing the two compilers might have been causing problems, so finally rebuilt everything with just gcc using this environment:

export HOMEBREW_USE_GCC=true
export CC=gcc-4.2
export CXX=g++-4.2
export FFLAGS=-ff2c
export HOMEBREW_BUILD_FROM_SOURCE=true

Extra notes: “man dyld” is very useful for manipulating shared libraries. In particular the DYLD_INSERT_LIBRARIES and DYLD_PRINT_LIBRARIES environment variables let you see and control which versions of libraries are being used.