Upgrading sqlite3 within Python 3

Python 3 has sqlite3 baked in. That’s great! But it has an old version of sqlite3; my Ubuntu 14.04 system with Python 3.4 has sqlite 3.8.2. I want sqlite 3.9, for the new JSON hotness.

There’s a pip-installable package called “pysqlite” that should be installable in a virtualenv. But it only supports python 2 and explicitly says “use the built-in sqlite3 for python 3”. Thanks but no thanks!

This Stack Exchange thread has details on recompiling python3 with your own sqlite3, but that’s more work than I want to do. Also it appears that python3 is just loading libsqlite3.so.0 from my Ubuntu install, so I should be able to upgrade it by either upgrading libsqlite in Ubuntu or building my own shared library. But the Ubuntu upgrade looks impossible without upgrading all of Ubuntu 14.04. (I couldn’t find a PPA from a quick search).

Compiling sqlite3 into a shared library is an exercise. The source is distributed in an idiosyncratic format and the version with a Makefile doesn’t build shared libraries. I ended up making something work by following the official docs and this note about the shared library.

gcc -shared -o libsqlite3.so -fPIC sqlite3.o -ldl -lpthread
ln -s libsqlite3.so libsqlite3.so.0
./sqlite3 -version
LD_LIBRARY_PATH=. /usr/bin/python3 -c 'import sqlite3; print(sqlite3.sqlite_version)'

It works! But now I’m nervous. Compiling sqlite3 correctly takes more research than I conducted. It has a whole lot of options! Also I get a bit worried about relying on LD_LIBRARY_PATH to override libraries. It’s totally fine to do of course, but it just seems too easy to forget to do it in some environment and suddenly things aren’t working consistently. If I were serious about doing this I think I’d build my own Ubuntu package for real, using Ubuntu’s Makefile as a starting point.