Thoughts on DuckDB's Crazy Grammar Thing

At Re:Invent last week, AWS announced DSQL, their new serverless SQL database. As a fan of distributed SQL databases I have been enjoying reading about the various architectural decisions in Marc Brooker’s blog. One thing I thought was fun to see was the first real retread of the Spanner atomic clocks trick since…Spanner? So I thought this would...

It is the job of modern programming languages to be amenable to abstractions. It should be easy for users to take a little bundle of functionality and reuse it, or build something more complex on top of it, or give it to someone else to use. A good programming language gives you a set of primitives and tools to combine them into more complex...

Follow me on Bluesky! A Log-Structured Merge tree, or LSM, is a popular data structure for storage engines. It’s what is used by RocksDB, which is sort of the poster child LSM. At a high level, the way an LSM works is that new writes get added to a memory-only index called a memtable and a durable file called a write-ahead log, or WAL. The...

Meta note: follow me on Bluesky if you are so inclined. I've been going through some historical stuff lately. Been on a history kick. We went to see the Robert Caro Power Broker exhibit at The New York Historical and it really got me in a mood to be digging through some old musty documents. The paper SEQUEL: A STRUCTURED ENGLISH QUERY LANGUAGE by...

Everything you need to know about query planning can be understood from this query: SELECT * FROM xy WHERE y = 3 ORDER BY x Imagine we have two indexes, one ordered on y, and one ordered on x. Then two possible plans suggest themselves: Plan 1: scan the y index, restricting it to the values of y such that y = 3, then sort the result by x. Plan 2:...