Moving forward with lolslackbot and Django

I’m very encouraged with how my little Django experiment has worked out. I’m ready to start using it with lolslackbot. A big breakthrough was realizing that I can start using Django as soon as I trust it not to corrupt my database. Keep the existing Python cron job in old code, not using Django, and then have a separate webapp that also uses the same database. The cron job and the webapp will be reading the same tables but in general not writing the same tables. It seems like a good transition plan.

Only now I really have to face the sqlite question. I’m confident that ultimately I need to be using PostgreSQL. I’m envisioning thousands of web users updating the database; there’s no way that works with sqlite’s database-level locking. Even with low traffic the cron job and the webapp will be stepping on each other. I need to switch. But when?

Do I switch databases first, porting my cron job code over to a Postgres-backed system? That seems really painful and not much fun. Lots of backend work with no visible features.

Or do I use Django first, having it talk to the sqlite database? The lock contention won’t be a real problem if I’m the only webapp user. It lets me build fun / useful features sooner. And it gives me some more experience with data modelling. I’m pretty sure I’m going to want to refactor the schema along with the Postgres update, it’d be better to do that after doing some of the webapp work. The risk with Django-first is I do a lot of sqlite-specific work that is ultimately wasted. But Django’s ORM insulates you from the underlying database pretty well, so maybe that doesn’t matter?

Also curious how the Django ORM is going to work for me. I use a couple of non-standard SQL things now. Mostly “insert or replace”, which is sqlite’s upsert-like extension. Does the ORM expose those? Also very curious how testing will work. sqlite is so simple when managing test environments. But I know the Django folks have thought this through.