TinyDB: is this sqlite for JSON?

Awhile back I wanted sqlite for JSON, some simple file-based JSON document store with a basic query interface. TinyDB looks promising. It’s a Python library. It lets you stuff a bunch of JSON documents into a file database and then query them. The query language is pretty simple, mostly “documents that have property X”, but that’s fine. The underlying file format looks like one giant JSON blob and may not be very efficient for updates, not sure.

I’ve used the JSON1 support in sqlite now. It’s pretty good.

I sure wish it were easy to marry jq to some simple JSON document store.

 

3 thoughts on “TinyDB: is this sqlite for JSON?

  1. What was missing from the SQLite extension for you when you tried it, good-enough querying? I poked around TinyDB a bit when I first ran across it and and it is as tiny as it says on the tin – the docs brag about it being 1200 lines of code with 40% of it being docstrings. This is attractive and admirable but the trouble is that getting [semi]structured data onto disc with even very basic durability and integrity guarantees is a tricky and fiddly business. I’m not entirely sure it can be done in pure Python at all, let alone in a few pages of code.

  2. Yeah I’m not sure I’d trust TinyDB to be ACID in the face of multiple concurrent writers. I didn’t look at the code though. I’m looking more for something I can just throw a bunch of JSON at as a one-off.

    There’s nothing inadequate about JSON1 in sqlite. It’s just a bit clumsy if you want the Mongo model. You still have to create a table with a schema, even if it’s one SQL column, and then the SQL query syntax isn’t very JSON friendly. It’s not bad though, maybe a little wrapper hiding the SQL part would be sufficient for the usability I’m thinking of.

  3. It seems deeply weird there is seemingly no particularly ergonomic API/library for this stuff, never mind one that lets you independently choose the method of persistence. Although perhaps the fact that there are things like urllib, urllib2, urllib3 (and then requests, built on top of the urllib3) suggests getting supposedly straightforward APIs right is tougher than it looks.

Comments are closed.