Need a database abstraction

My little lolslackbot project has grown big. 11 different tables! I’m hand-writing ORM code now, silly things like “class Person” which wraps a row for the people table. The meat of the work is still hand-crafted SQL with proper relational queries, but I’m increasingly doing some really basic “get me the row with property X” and writing it all by hand is tedious. And oh boy I added some caching using functools.lru_cache, which I know from experience at Google is the beginning of the end for understandable correctness. And I continue to write increasingly elaborate schema migration scripts as I upgrade stuff.

It’s all kind of a mess. I wish now I’d started using some Python ORM tool. I still think ORMs are kind of evil, but they are a useful evil.

I’m out of date on where Python is but my impression is everyone likes SQLAlchemy. The basic tools are just a really nice upgrade from the basic Python database driver stuff. And the ORM looks like it’s made the right choices, at least based on what I read.

But do I want to port all this code over? Hell no. Hrm… I’m accumulating technical debt. I also need to wrap all my Python code in a proper module structure, and set up a real virtualenv, and…

At least I have a test suite. Although Guido help me if I ever have to regenerate the test dataset.


2 thoughts on “Need a database abstraction

  1. These are great reading and I hope you have the time and inclination to keep writing them down. ‘What is stupid shit that happens when you write software in the small that turns out to be not that small’.

  2. Thanks! Truthfully this is still pretty small, a couple hundred users and one database transaction a second. Nevertheless I’m definitely in the realm of needing higher level abstractions now.

Comments are closed.