Hacking SaaS #21 - Design of Cloud Native Databases
Hacking SaaS #21 - Design of Cloud Native Databases
Fair warning: this edition isn’t fluffy. Get ready for a collection of deep-dives into the most interesting architectures currently running at scale.
Over the last few months, I read a lot of papers, docs, and in-depth presentations that describe the design of some of the most successful and popular cloud-native databases. I think this collection will be useful or at least fascinating to any senior software developer or architect.
Most of the links here are worth an hour of your time, in a nice quiet spot, with your favorite beverage. I’d also put some light jazz for the atmosphere.
If you prefer lighter, shorter, more practical, or more beginner-friendly content - I promise to make it up to you in the next edition.
Relational Databases
Massively Scaling MySQL Using Vitess (Video, slides, and transcription or presentation)
The Snowflake Elastic Data Warehouse (Paper)
Cockroach DB architecture in 5 parts + their cloud architecture (Documentation + a course)
Yugabyte architecture (26 articles in 9 sections)
AlloyDB for PostgreSQL under the hood: Columnar engine (Blog)
Amazon Aurora:
Design considerations for high throughput cloud-native relational databases (Paper)
On avoiding distributed consensus for I/Os, commits, and membership changes (Paper)
Google’s Spanner / F1: (all papers)
Non-relational and possibly non-databases
Amazon DynamoDB: A Scalable, Predictably Performant, and Fully Managed NoSQL Database Service (Paper)
The Cloud-Native Chasm: Lessons Learned from Reinventing Apache Kafka® as a Cloud-Native, Online Service (Paper)
Schema-Agnostic Indexing with Azure DocumentDB (Paper)
Amazon S3:
Using Lightweight Formal Methods to Validate a Key-Value Storage Node in Amazon S3 (Paper)
Building and Operating a Pretty Big Storage System (My Adventures in Amazon S3) (Video of presentation)
AWS re: Invent 2019: Beyond eleven nines: Lessons from Amazon S3 culture of durability (Video of presentation)
On SaaS Developer Youtube
My interview with one of our SaaS Developer Heroes - Jeffrey Sherman.
We discussed his “Never Rewrite” philosophy and how it applies even to early-stage companies building their MVPs. We also discuss the critical design decisions engineers have to make in the early days of building the product: Tabs or spaces, pooled tenants or siloed, Micro-services or monolith... And who makes these decisions? And Jeffrey shared funny stories related to these decisions and other common scenarios when building SaaS.