
Building a Key-Value Database on S3 in Go
Isledb is an embedded Go key-value database library on S3 using LSM trees.
- Writes are batched in the memtable and then flused into SST which gets uploaded to object store.
- A CAS conditional(e-tag) commits of Current Manifest is made with fencing for the SST to be made viisible for the reader.
- Compaction can run on seperate process.
- Reader reads directly from s3 object store
Source Code: https://github.com/ankur-anand/isledb
Blog: Outlines all the design in detail and various nuances and decisions: https://medium.com/@ankur_anand/building-a-key-value-database-on-s3-in-go-fb08d040f0af