Hacking SaaS #7 - Mostly Platforms
I hear that platform engineering is trending again. Here are few interesting posts that caught my attention in the last two weeks.
Charity Majors explains why platform engineering is the future of ops. And as often happens, after she shares her point of view, it seems obvious and everyone says that this was always obviously true. Its a long-ish post with lots of gems like this:
The maze of APIs and SDKs and components out there is simply bewildering, even for an experienced ops hand. Before long, it becomes someone’s job to make good decisions, pick a suite of compute and storage options that serve the team’s needs, and write some tooling that pulls everything into a coherent whole—which, at a minimum, lets you:
Run tests and generate new artifacts
Deploy artifacts, version them, and roll back
Instrument, monitor, and debug
Store data somewhere, manage schemas and migrations
Adjust capacity as needed
Define and commit all components (and their relationships) as code
Once these are built, it should be trivial for an engineer to come along and spin up a new service using templates and components from existing services. It should be much simpler and easier to use the blessed paths than anything else, and there should be friction if you go off the beaten path.
Congratulations! You’ve just been platformed . One of the key principles of any developer platform is that it should be easy to do the right things, and hard to do the wrong things.
Usually, before you have been platformed, you were microserviced. VMWare Tanzu team published a great series on designing microservices that covers lifecycles, scalability, failures, dependencies and a lot more. You should read the whole thing, starting with the one about what should and shouldn’t be a microservice.
These days, you can't swing a dry erase marker without hitting someone talking about microservices…
There are many good reasons to use a microservices architecture. But there are no free lunches, and the positives of microservices come with added complexity. Teams should happily take on that complexity ... provided the application in question benefits from the upside of microservices….
To rid the world of surplus microservices, we present this list to help focus your efforts. Read through the following principles and ask if the application in question benefits from a given principle. If you answer “yes” for one or more of the following principles, the feature is a good candidate to be a microservice. If you answer “no” for every principle, you are likely introducing accidental complexity into your system.
A special case of a platform is the data platform. Whether you have a centralized data team or implemented the data mesh socio-technical architecture, you probably still have a data platform of some sort. And if contracts / schema are not part of this platform, you are probably doing it wrong. I may be preaching to the choir here, but Jake from buz.dev explains the benefits of contract-based data platform really well. His blog has many gems, but I’m partial to JSON Schema these days, and this one really hit home:
Schematizing data upfront means data discovery and documentation writes itself. Data assets are discoverable as soon as schemas are deployed - before the data actually starts flowing.
Schematization mechanisms like JSON Schema also get pretty meta, so it's easy to add annotation such as:
"These properties contain PII"
"These properties should be tokenized"
"X person on Y team owns this schema"
"This is version
1.4
. Here's how this data has evolved from1.0
."This class of metadata is a CISO's dream.
For a slightly different perspective on the topic, there’s a talk I gave at QCon NYC many years ago: Streaming Microservices - contracts and compatibility (and a related interview). The stories and examples are evergreen, but today I’d take a softer stance on the schema definition language. JSON Schema became pretty great while I was looking the other way.
Data contracts aside, most services communicate via REST APIs, and in the SaaS world - we often offer our services to the world through APIs on the public internet. Hacking SaaS #6 has good resources around schema and compatibility for those. This week we also have an interview on the SaaS Developer Youtube (hit that subscribe button!) with an active SaaS Developer community member - Buchi Reddy. We talked about API Security, bad things that happen to good APIs, the “shift left” trend in application security and whether you still need a security team.
Join our community Slack to continue the conversation:
While not exactly about platforms, I recommend reading this informative thread about caching, bimodality and little known aspects of DynamoDB architecture. While we don’t all work at Amazon scale, bimodality is a risk in every architecture and worth being familiar with it: