What every SaaS developer should read on vacation

Last Hacking SaaS for 2022! Get inspired for 2023 with short videos on tech trends and longer articles that you can dive into during the holiday break.

Happy Holidays to all SaaS Developers! I hope you all feel good with everything you shipped this year, and are getting ready for your well deserved holidays.

I typically do the most learning during the holidays. The break is an opportunity to step away from the pile of urgent tickets and deadlines, and to reflect, learn something new, think deeply about new concepts and perhaps hack a quick POC around an exciting idea.

To help with this, I collected more content than usual, and with some bias toward longer content or deeper ideas - the one that you may not have time to read and digest during a busy workday. Because every SaaS Developer is passionate about different aspects of SaaS, I tried to have good links for every taste.

But before we dig deep, lets start with two short videos that review the important trends for SaaS developers in 2022 / 2023:

and now, lets get to the meat!

Databases

Some opinionated thoughts on SQL databases starts as a standard rant, but don’t let this fool you, this blog goes much deeper discusses serious concerns and offers ways to mitigate them.

The post also links to a story about a mysterious database performance issue. I love learning from other people’s experience, especially when they are that well written.

What do you do if you want to pool all your SaaS tenants into one database, but you also want to store encrypted data for them? securely? And don’t want to manage a separate key per tenant? This blog from AWS offers a cost-effective and secure solution that only slightly resembles a rube goldberg machine.

And speaking of tenants, Shopify shared some of their store-management techniques, and they apply to multi-tenant databases too. Learn how they handle highly loaded shards, and how they route traffic to the correct backend and data store.

Serverless

If you haven’t read the classic paper “Serverless Computing: One Step Forward, Two Steps Back”, definitely worth reading. It describes a lot of the gaps in Function-as-a-Service (FaaS) and set direction for the industry.

Stateful FaaS became a bit of a holy grail of cloud computing. Especially when considering the problem of multi-region and edge functions. Almost every cloud vendor and CDN vendor agrees that there is a problem and has a different approach to solving it. The Cloudburst Paper from Berkley’s RiseLab suggests an approach that provides causal consistency with very low latency using both distributed databases and smart caches.

Smart caches are shaping to be a major 2023 trend, so worth reading about it. In addition to this paper, there is also Venice’s approach, Cloudflare’s Worker KV, and Noria (now being commercialized as ReadySet). Let me know if there are others that I should follow.

HTTP - Where networks and SaaS meet

HTTP/2 is fairly ubiquitous now, not just in the backend but also in browsers. You may have heard that it is faster? In this case, you may want to read a blog that explains exactly under what conditions HTTP/2 will be faster. Even if you never implemented an HTTP/2 server, some cloud load balancers and CDNs will automatically serve your content with HTTP/2 to clients that support it. So understanding the implications is useful anyway.

And while we are busting myths here, you probably want to read about the surprising case where making an API call over the network network is faster than reading from local cache.

Platforms

Considering that it started at Google and was called “operating system for the data center”, Kubernetes can be surprisingly challenging to scale. This blog shares lessons learned from a 7500 node K8s cluster.

We all had cases where we moved an application to a larger machine, but did not see the performance improvements we expected. Some engineers would shrug, mumble something about contention and leave it as is, knowing that they are leaving literal money on the table. Netflix engineers on the other hand, dig deep all the way into the CPU cache lines and the assembly code of the JVM. Great to read and their techniques are useful to any engineering team that cares about performance and cloud costs.

Software engineering advice

It is very easy to guess that if you are building a SaaS product, your production database includes a table called “Users”. But, did you model it correctly? Chances are good that you didn’t. In our experience, most developers get it wrong at first. This blog points out all the ways where you could be wrong in how you model users.

My twitter feed is full of hot takes about debugging, but the basic debugging tactics didn’t change in decades. Having a good set of tactics will immediately make anyone a better engineer.

Testing is another software engineering skill that never gets old. And there are always new techniques to learn. I’m sure you all know how to write tests (although many developers should learn about microbenchmarks, property-based testing and deterministic simulation testing). But, how you do know your test coverage is any good? Running Codecov and looking at the % is not enough. So many times I discovered, while debugging, that the test didn’t actually test what we thought it tests. What do we do? Check out Java mutation tools - they make semantically correct but random changes to your code. If your code changes but the tests still pass, you should be a bit concerned.

Thats it for now, and looking forward to all of us becoming even better SaaS Developer next year!