I would really like to send this article out to all the developers in my small company (only 120+ people, about 40 dev & test) but the political path has been chosen and the new shiny tech has people entranced.
What we do (physics simulation software) doesn’t need all the complexity (in my option as a long time software developer & tester) and software engineering knowledge that splitting stuff into micro services require.
Only have as much complexity as you absolutely need, the old saying “Keep it simple, stupid” still has a lot of truth.
But the path is set, so I’ll just do my best as an individual contributor for the company and the clients who I work with.
I started making the case for organizational efficiency rather than a technical argument. Demonstrating where the larger number of people and teams necessary to make a decision and a change and how that impacts the amount of time to ship new features has been more effective IME.
If you have workloads with different shapes, microservices make sense.
If not, do the monolith thing as long as you can.
But if you're processing jobs that need hand off to a GPU, just carve out a service for it. Stop lamenting over microservices.
If you've got 100+ engineers and different teams own different things, try microservices. Otherwise, maybe keep doing the monolith.
If your microservice is as thin as leftpad.js and hosts only one RPC call, maybe don't do that. But if you need to carve out a thumbnailing service or authC/authZ service, that's a good boundary.
There is no "one size fits all" prescription here.
I wonder, at which point is a service getting called microservice? The team-sized service advocated by the usual argument does not sound that "micro" to me - but is most of the times the right size.
The definitional size I've read and heard is that you team could (with the benefit of hindsight) be able to reimplement a microservice in 2 weeks. That sounds fairly extreme but a month seems within reason to me.
The other key difference between microservices and other architectures is that each microservice should do its primary function (temporarily) without hard dependencies, which basically means having a copy of the data that's needed. Service Oriented Architecture doesn't have this as one of its properties which is why I think of it as a mildly distributed monolith. "Distributed monolith" is the worst thing you could call a set of microservices--all the pain without the gains.
I'm helping a company get out of legacy hell right now. And instead of saying we need microservices, let's start with just a service oriented architecture. That would be a huge step forward.
Most companies should be perfectly fine with a service oriented architecture. When you need microservices, you have made it. That's a sign of a very high level of activity from your users, it's a sign that your product has been successful.
Don't celebrate before you have cause to do so. Keep it simple, stupid.
Service oriented architecture seems like a pretty good idea.
I've seen a few regrettable things at one job where they'd ended up shipping a microservice-y design but without much thought about service interfaces. One small example: team A owns a service that runs as an overnight job making customer specific recommendations that get written to a database, and then team B owns a service that surfaces these recommendations as a customer-facing app feature and directly reads from that database. It probably ended up that way as team A had the data scientists and team B had the app backend engineers for that feature and they had to ship something and no architect or senior engineer put their foot down about interfaces.
That'd be pretty reasonable design if team A and team B were the same team, so they could regard the database as internal with no way to access it without going through a service with a well defined interface. Failing that, it's hard to evolve the schema of the data model in the DB without a well defined interface you can use to decouple implementation changes from consumers and where the consuming team B have their own list of quarterly priorities.
Microservices & alternatives aren't really properties of the technical system in isolation, they also depend on the org chart & which teams owns what parts of the overall system.
SOA: pretty good, microservices: probably not a great idea, microservices without SOA: avoid.
For anyone unfamiliar with SOA, there's a great sub-rant in Steve Yegge's 2011 google platforms rant [1][2] focusing on Amazon's switch to service oriented architecture.
> And instead of saying we need microservices, let's start with just a service oriented architecture.
I think the main reason microservices were called “microservices” and not “service-oriented architecture” is that they were an attempt to revive the original SOA concept when “service-oriented architecture” as a name was still tainted by association to a perceived association with XML and the WS-* series of standard (and, ironically, often systems that supported some subset of those standards for interaction despite not really applying the concepts of the architectural style.)
You need multiple services whenever the scaling requirements of two components of your system are significantly different. That's pretty much it. These are often called micro services, but they don't have to actually be "micro"
I came here to say the same. If you’re arguing either for or against microservices you’re probably not thinking about the problem correctly. Running one big service may make sense if your resource needs are pretty uniform. Even if they’re not you need to weight the cost of adding complexity vs the cost of scaling some things prematurely or unnecessarily. Often this is an acceptable precursor to splitting up a process.
There's one thing I've learned about microservices. If you've ever gone down the path of making them, failing and making them again until they all worked as they should with the desired 9's of uptime, then you'll only want to make them if it's really the right thing to make. It's not worth the effort otherwise.
So no I don't want microservices (again), but sometimes it's still the right thing.
I like goldilocks services, as big or as small as actually makes sense for your domain/resource considerations, usually no single http endpoint services in sight.
Once upon a time, that's what a microservice was. A monolith was the company's software all in one software package.
I think what changed things is FAAS came along and people started describing nanoservices as microservices which created really dumb decisions.
I've worked on a true monolith and it wasn't fun. Having your change rolled back because another team made a mistake and it was hard to isolate the two changes was really rough.
What's the performance trade-off of something like this over containerization? I have heard of an operating system that runs WASM (https://github.com/JonasKruckenberg/k23).
highly depends on the wasm runtime we're running things on. I haven't seen any good recent benchmarks (as in the past few years). But, if I remember right wasmer is putting together some and trying to automate the results for them.
I don't want microservices; I want an executable. Memory is shared directly, and the IDE and compiler know about the whole system by virtue of it being integrated.
Probably works OK for a small project with a close knit team of skilled contributors where there's some well defined structure and everyone has sufficient high level understanding of that structure to know what kinds of dependencies are or are not healthy to have.
But, unless you have some way of enforcing that access between different components happens through some kind of well defined interfaces, the codebase may end up very tightly coupled and expensive or impractical to evolve and change, if shared memory makes it easy for folks to add direct dependencies between data structures of different components that shouldn't be coupled.
I loved uberjars back when I was writing Scala. I don't miss much about the JVM, but I really miss having a single executable I could just upload and run without having to pay attention to the environment on the host machine.
Heavier weight? Yes. Slower? Should be the same performance. Unless you are on a non-linux host, then there is no docker penalty.
The only time I can think where a JVM might be faster is if you have a multi-tenant setup. In that case, the JVM can be more effective with the GC vs having multiple JVMs running.
"Traditional" three-tier, where you have a web server talking to an application server talking to a database server, seems like overkill; I'd get rid of the separate application tier.
If your tiers are browser, web API server, database: then three tiers still makes sense.
in my opinion "you need microservices" peaked around 2018-2019 ... does nowadays someone think that, apart from when you reach certain limits and specific contexts, they are a good idea?
Half of the jobs I'm applying to have microservices in the description, much more often than, say, REST or Boot, so somebody definitely thinks they're a general solution to something.
I feel like this has been beaten to death and this article isn't saying much new. As usual the answer is somewhere in the middle (what the article calls "miniservices"). Ultimately
1. Full-on microservices, i.e. one independent lambda per request type, is a good idea pretty much never. It's a meme that caught on because a few engineers at Netflix did it as a joke that nobody else was in on
2. Full-on monolith, i.e. every developer contributes to the same application code that gets deployed, does work, but you do eventually reach a breaking point as either the code ages and/or the team scales. The difficulty of upgrading core libraries like your ORM, monitoring/alerting, pandas/numpy, etc, or infrastructure like your Java or Python runtime, grows superlinearly with the amount of code, and everything being in one deployed artifact makes partial upgrades either extremely tricky or impossible depending on the language. On the operational and managerial side, deployments and ownership (i.e. "bug happened, who's responsible for fixing?") eventually get way too complex as your organization scales. These are solvable problems though, so it's the best approach if you have a less experienced team.
3. If you're implementing any sort of SoA without having done it before -- you will fuck it up. Maybe I'm just speaking as a cynical veteran now, but IMO lots of orgs have keen but relatively junior staff leading the charge for services and kubernetes and whatnot (for mostly selfish resume-driven development purposes, but that's a separate topic) and end up making critical mistakes. Usually some combination of: multiple services using a shared database; not thinking about API versioning; not properly separating the domains; using shared libraries that end up requiring synchronized upgrades.
There's a lot of service-oriented footguns that are much harder to unwind than mistakes made in a monolithic app, but it's really hard to beat SoA done well with respect to maintainability and operations, in my opinion.
Re 1: I like Matt Ranney's take on it, where he says microservices are a form of technical debt – they let you deploy faster and more independently in exchange for an overall more complex codebase.
This makes it clear when you might want microservices: you're going through a period of hypergrowth and deployment is a bigger bottleneck than code. This made sense for DoorDash during covid, but that's a very unusual circumstance
I see a lot of value in spinning up microservices where the database is global across all services (and not inside the service) but I struggle more to see the value of separate core transactional databases for separate services unless/until the point where two separate parts of the organizations are almost two separate companies that cannot operate as a single org/single company. You lose data integrity, joining ability, one coherent state of the world, etc.
The main time I can see this making sense is when the data access patterns are so different in scale and frequency that they're optimizing for different things that cause resource contention, but even then, my question would become do you really need a separate instance of the same kind of DB inside the service, or do you need another global replica/a new instance of a new but different kind of DB (for example Clickhouse if you've been running Postgres and now need efficient OLAP on large columnar data).
Once you get to this scale, I can see the idea of cell-based architecture [1] making sense -- but even at this point, you're really looking at a multi-dimensionally sharded global persistence store where each cell is functionally isolated for a single slice of routing space. This makes me question the value of microservices with state bound to the service writ large and I can't really think of a good use case for it.
> I see a lot of value in spinning up microservices where the database is global across all services (and not inside the service)
This issue with this is schema evolution. As a very simple example, let's say you have a User table, and many microservices accessing this table. Now you want to add an "IsDeleted" column to implement soft deletion; how do you do that? First you need to add the actual column to the database, then you need to go update every single service which queries that table and ensure that it's filtering out IsDeleted=True, deploy all those services, and only then can you actually start using the column. If you must update services in lockstep like this, you've built a distributed monolith, which is all of the complexity of microservices with none of the benefits.
A proper service-oriented way to deal with this is have a single service with control of the User table and expose a `GetUsers` API. This way, only one database and its associated service needs to be updated to support IsDeleted. Because of API stability guarantees--another important guarantee of good SoA--other services will continue to only get non-deleted users when using this API, without any updates on their end.
> You lose data integrity, joining ability, one coherent state of the world, etc.
You do lose this! And it's one of the tradeoffs, and why understanding your domain is so important for doing SoA well. For subsets of the domain where data integrity is important, it should all be in one database, and controlled by one service. For most domains, though, a lot of features don't have strict integrity requirements. As a concrete though slightly simplified example, I work with IoT time-series data, and one feature of our platform is using some ML algorithms to predict future values based on historical trends. The prediction calculation and storage of its results is done in a separate service, with the results being linked back via a "foreign key" to the device ID in the primary database. Now, if that device is deleted from the primary database, what happens? You have a bunch of orphaned rows in the prediction service's database. But how big of a deal is this actually? We never "walk back" from any individual prediction record to the device via the ID in the row; queries are always some variant of "give me the predictions for device ID 123". So the only real consequence is a bit of database bloat, which can be resolved via regularly scheduled orphan checking processes if it's a concern.
It's definitely a mindshift if you're used to a "everything in one RDBMS linked by foreign keys" strategy, but I've seen this successfully deployed at many companies (AWS, among others).
I found a different benefit to micro services — AI understands them and context matters. Monolithic app confuse ai where micro services enables them to be far more effective.
The other problem is that very very few people actually know how to design a microservice based architecture. I've worked with half a dozen different teams who claim they're building microservices, but when you look at the system it's just a giant distributed monolith. Most of them are people who worked in legacy code bases, and while they like the idea of microservices, they can't let go of those design patterns. So they do the exact same thing but just out everything behind network calls. Drives me absolutely fucking nuts
Another good use case for a microservice - if you are going to have to change the compute size for your monolith just to accommodate the new functionality.
I had an architect bemoan the suggestion we use a microservice, until he had to begrudgingly back down when he was told that the function we were talking about (Running a CLIP model) would mean attaching a GPU to every task instance.
We've removed/merged most of the unnecessary services. The ones left have operational needs to stay separate.
The current hell is x years of undisciplined (in terms of perf and cost) new ORM code being deployed (SQLAlchemy). We do an insane number of read queries per second relative to our usage.
I honestly think the newish devs we have hired don't understand SQL at all. They seem to think of it as some arcane low level thing people used in the 80s/90s.
What I want is a lightweight infrastructure for macro-services. I want something to handle the user and machine-to-machine authentication (and maybe authorization).
I don't WANT the usual K8s virtual network for that, just an easy-to-use module inside the service itself.
You should be able to spin up everything localy in a docker-compose container.
> What I want is a lightweight infrastructure for macro-services. I want something to handle the user and machine-to-machine authentication (and maybe authorization).
> I don't WANT the usual K8s virtual network for that, just an easy-to-use module inside the service itself.
K8s makes sense if you have a dedicated team (or atleast engineer) and if you really need need the advanced stuff (blue/green deployments, scaling, etc). Once it's properly setup it's actually a very pleasant platform.
If you don't need that Docker (or preferable Podman) is indeed the way to go. You can actually go quite far with a VPS or a dedicated server these day. By the time you outgrow the most expensive server you can (reasonable) buy you can probably afford the staff to roll out a "big boy" infrastructure.
I tried to use K8s several times, and I just can't make it work. It's fine as a deployment platform, but I just can't justify its complexity for local development.
We're using Docker/Podman with docker-compose for local development, and I can spin up our entire stack in seconds locally. I can attach a debugger to any component, or pull it out of the Docker and just run it inside my IDE. I even have an optional local Uptrace installation for OTEL observability testing.
My problem is that our deployment infrastructure is different. So I need to maintain two sets of descriptions of our services. I'd love a solution that would unify them, but so far nothing...
I wouldn't use K8s for local development unless you have some system where there is a dev cluster and you can route traffic for particular pod to your local workstation.
Docker Compose for local development is fine. If your K8s setup is crazy complex that you need to test it locally, please stop.
Tilt? Skaffold? configuration isn't free, but a debugger on a local k8 cluster that's at least somewhat representative of prod is pretty handy once you do.
I tried Tilt, but it's still too complicated. For example, we have a computer vision model that is a simple Python service. When developing on macOS, it's not possible to use GPUs inside containers, so we need to run it locally on the host.
It's trivial with my current setup, but not really possible with Tilt.
I know this pain, though we’re running nomad not k8s as our cluster control plane. But local devs are stuck with docker-compose, so 2 different configurations for running locally versus in the production environment.
Unless you need horizontal scalability or clustering, Compose + Terraform is all you need.
With Compose, you get proper n-tier application containerization with immutability. By adding an infrastructure-as-code tool such as Terraform to abstract your IT environment, you can deploy your application on-premises, in the cloud, or at a customer site with a single command.
For clustering needs, there’s Incus, and finally, Kubernetes for very fast scalability (<mn), massive deployments on large clusters, cloud offloading, and microservices.
Almost nobody truly needs the complexity of Kubernetes. The ROI simply isn’t there for the majority of use cases.
In my experience basically everything being good in software is downstream of good data modelling.
It's partly why I've realised more over time that learning computer science fundamentals actually ends up being super valuable.
I'm not talking about anything particularly deep either, just the very fundamentals you might come across in year one or two of a degree.
It sort of hooks back in over time as you discover that these people decades ago really got it and all you're really doing as a software engineer is rediscovering these lessons yourself, basically by thinking there's a better way, trying it, seeing it's not better, but noticing the fundamentals that are either being encouraged or violated and pulling just those back out into a simpler model.
I feel like that's mostly what's happened with the swing over into microservices and the swing back into monoliths, pulling some of the fundamentals encouraged by microservices back into monolith land but discarding all the other complexities that don't add anything.
I feel that if you have multiple sets of application logic that need to access the same data, there should be an internal API between them and the database that keeps that access to spec.
I would really like to send this article out to all the developers in my small company (only 120+ people, about 40 dev & test) but the political path has been chosen and the new shiny tech has people entranced.
What we do (physics simulation software) doesn’t need all the complexity (in my option as a long time software developer & tester) and software engineering knowledge that splitting stuff into micro services require.
Only have as much complexity as you absolutely need, the old saying “Keep it simple, stupid” still has a lot of truth.
But the path is set, so I’ll just do my best as an individual contributor for the company and the clients who I work with.
I started making the case for organizational efficiency rather than a technical argument. Demonstrating where the larger number of people and teams necessary to make a decision and a change and how that impacts the amount of time to ship new features has been more effective IME.
I thought microservices were old by now, which is why this kind of articles are finally appearing.
I would really like to send this article back in time 11 years
If you have workloads with different shapes, microservices make sense.
If not, do the monolith thing as long as you can.
But if you're processing jobs that need hand off to a GPU, just carve out a service for it. Stop lamenting over microservices.
If you've got 100+ engineers and different teams own different things, try microservices. Otherwise, maybe keep doing the monolith.
If your microservice is as thin as leftpad.js and hosts only one RPC call, maybe don't do that. But if you need to carve out a thumbnailing service or authC/authZ service, that's a good boundary.
There is no "one size fits all" prescription here.
I wonder, at which point is a service getting called microservice? The team-sized service advocated by the usual argument does not sound that "micro" to me - but is most of the times the right size.
The definitional size I've read and heard is that you team could (with the benefit of hindsight) be able to reimplement a microservice in 2 weeks. That sounds fairly extreme but a month seems within reason to me.
The other key difference between microservices and other architectures is that each microservice should do its primary function (temporarily) without hard dependencies, which basically means having a copy of the data that's needed. Service Oriented Architecture doesn't have this as one of its properties which is why I think of it as a mildly distributed monolith. "Distributed monolith" is the worst thing you could call a set of microservices--all the pain without the gains.
Good point. They would not have been as popular if they were called "multi-services" or "team partitioned apps".
I'm helping a company get out of legacy hell right now. And instead of saying we need microservices, let's start with just a service oriented architecture. That would be a huge step forward.
Most companies should be perfectly fine with a service oriented architecture. When you need microservices, you have made it. That's a sign of a very high level of activity from your users, it's a sign that your product has been successful.
Don't celebrate before you have cause to do so. Keep it simple, stupid.
Service oriented architecture seems like a pretty good idea.
I've seen a few regrettable things at one job where they'd ended up shipping a microservice-y design but without much thought about service interfaces. One small example: team A owns a service that runs as an overnight job making customer specific recommendations that get written to a database, and then team B owns a service that surfaces these recommendations as a customer-facing app feature and directly reads from that database. It probably ended up that way as team A had the data scientists and team B had the app backend engineers for that feature and they had to ship something and no architect or senior engineer put their foot down about interfaces.
That'd be pretty reasonable design if team A and team B were the same team, so they could regard the database as internal with no way to access it without going through a service with a well defined interface. Failing that, it's hard to evolve the schema of the data model in the DB without a well defined interface you can use to decouple implementation changes from consumers and where the consuming team B have their own list of quarterly priorities.
Microservices & alternatives aren't really properties of the technical system in isolation, they also depend on the org chart & which teams owns what parts of the overall system.
SOA: pretty good, microservices: probably not a great idea, microservices without SOA: avoid.
For anyone unfamiliar with SOA, there's a great sub-rant in Steve Yegge's 2011 google platforms rant [1][2] focusing on Amazon's switch to service oriented architecture.
[1] https://courses.cs.washington.edu/courses/cse452/23wi/papers... [2] corresponding HN thread from 2011 https://news.ycombinator.com/item?id=3101876
> And instead of saying we need microservices, let's start with just a service oriented architecture.
I think the main reason microservices were called “microservices” and not “service-oriented architecture” is that they were an attempt to revive the original SOA concept when “service-oriented architecture” as a name was still tainted by association to a perceived association with XML and the WS-* series of standard (and, ironically, often systems that supported some subset of those standards for interaction despite not really applying the concepts of the architectural style.)
You need multiple services whenever the scaling requirements of two components of your system are significantly different. That's pretty much it. These are often called micro services, but they don't have to actually be "micro"
I came here to say the same. If you’re arguing either for or against microservices you’re probably not thinking about the problem correctly. Running one big service may make sense if your resource needs are pretty uniform. Even if they’re not you need to weight the cost of adding complexity vs the cost of scaling some things prematurely or unnecessarily. Often this is an acceptable precursor to splitting up a process.
There's one thing I've learned about microservices. If you've ever gone down the path of making them, failing and making them again until they all worked as they should with the desired 9's of uptime, then you'll only want to make them if it's really the right thing to make. It's not worth the effort otherwise.
So no I don't want microservices (again), but sometimes it's still the right thing.
I like goldilocks services, as big or as small as actually makes sense for your domain/resource considerations, usually no single http endpoint services in sight.
Once upon a time, that's what a microservice was. A monolith was the company's software all in one software package.
I think what changed things is FAAS came along and people started describing nanoservices as microservices which created really dumb decisions.
I've worked on a true monolith and it wasn't fun. Having your change rolled back because another team made a mistake and it was hard to isolate the two changes was really rough.
I don't want microservices, I think what I really want is self contained WebAssembly modules!
What's the performance trade-off of something like this over containerization? I have heard of an operating system that runs WASM (https://github.com/JonasKruckenberg/k23).
highly depends on the wasm runtime we're running things on. I haven't seen any good recent benchmarks (as in the past few years). But, if I remember right wasmer is putting together some and trying to automate the results for them.
Unironically this
On the theme of several other responders:
I don't want microservices; I want an executable. Memory is shared directly, and the IDE and compiler know about the whole system by virtue of it being integrated.
Probably works OK for a small project with a close knit team of skilled contributors where there's some well defined structure and everyone has sufficient high level understanding of that structure to know what kinds of dependencies are or are not healthy to have.
But, unless you have some way of enforcing that access between different components happens through some kind of well defined interfaces, the codebase may end up very tightly coupled and expensive or impractical to evolve and change, if shared memory makes it easy for folks to add direct dependencies between data structures of different components that shouldn't be coupled.
I love the idea that I can compile all my functionality including HTML templates, javascript, and CSS into a single albeit huge Golang binary.
I have never done this yet.
But I love the idea of it.
You can already do this with Deno Compile
https://deno.com/blog/deno-compile-executable-programs
This is what I'm doing with my side project! A set of personal smart picture frames for me and my partner. One executable does:
- the uploader API
- the uploader UI
- the frame API
- the frame UI
UIs are SSG'd with solid-js and solid-start then served with gin.
It's really fun.
I loved uberjars back when I was writing Scala. I don't miss much about the JVM, but I really miss having a single executable I could just upload and run without having to pay attention to the environment on the host machine.
That's essentially the role docker serves now. Everything you need to run in 1 single image.
Yeah, but building a docker image tends to be a lot heavier weight and slower, in my experience, than uploading a single jar
Heavier weight? Yes. Slower? Should be the same performance. Unless you are on a non-linux host, then there is no docker penalty.
The only time I can think where a JVM might be faster is if you have a multi-tenant setup. In that case, the JVM can be more effective with the GC vs having multiple JVMs running.
Great nick!
microservices were an effect of the ZIRP era. you literally have places like Monzo bragging that they've 3 microservices for each engineer.
3 tier architecture proves time and time again to be robust for most workloads.
1 micro-service per pizza sized team seems to work pretty well.
Put it into a monorepo so the other teams have visibility in what is going on and can create PRs if needed.
Uh? You eat less than a pizza per person?
It's a reference to Amazon's statement that teams should never grow larger than a team that you can feed with 2 (large) pizza's.
The optimum is probably closer to 1 than to 2.
To be fair pizzas are quite easy to scale from small kid sizes up to enough for several persons.
But it is a bit sad that the poster apparently never bought a pizza just for themselves.
8x engineer
Certainly no more than three tiers.
"Traditional" three-tier, where you have a web server talking to an application server talking to a database server, seems like overkill; I'd get rid of the separate application tier.
If your tiers are browser, web API server, database: then three tiers still makes sense.
in my opinion "you need microservices" peaked around 2018-2019 ... does nowadays someone think that, apart from when you reach certain limits and specific contexts, they are a good idea?
Half of the jobs I'm applying to have microservices in the description, much more often than, say, REST or Boot, so somebody definitely thinks they're a general solution to something.
I feel like this has been beaten to death and this article isn't saying much new. As usual the answer is somewhere in the middle (what the article calls "miniservices"). Ultimately
1. Full-on microservices, i.e. one independent lambda per request type, is a good idea pretty much never. It's a meme that caught on because a few engineers at Netflix did it as a joke that nobody else was in on
2. Full-on monolith, i.e. every developer contributes to the same application code that gets deployed, does work, but you do eventually reach a breaking point as either the code ages and/or the team scales. The difficulty of upgrading core libraries like your ORM, monitoring/alerting, pandas/numpy, etc, or infrastructure like your Java or Python runtime, grows superlinearly with the amount of code, and everything being in one deployed artifact makes partial upgrades either extremely tricky or impossible depending on the language. On the operational and managerial side, deployments and ownership (i.e. "bug happened, who's responsible for fixing?") eventually get way too complex as your organization scales. These are solvable problems though, so it's the best approach if you have a less experienced team.
3. If you're implementing any sort of SoA without having done it before -- you will fuck it up. Maybe I'm just speaking as a cynical veteran now, but IMO lots of orgs have keen but relatively junior staff leading the charge for services and kubernetes and whatnot (for mostly selfish resume-driven development purposes, but that's a separate topic) and end up making critical mistakes. Usually some combination of: multiple services using a shared database; not thinking about API versioning; not properly separating the domains; using shared libraries that end up requiring synchronized upgrades.
There's a lot of service-oriented footguns that are much harder to unwind than mistakes made in a monolithic app, but it's really hard to beat SoA done well with respect to maintainability and operations, in my opinion.
Re 1: I like Matt Ranney's take on it, where he says microservices are a form of technical debt – they let you deploy faster and more independently in exchange for an overall more complex codebase.
This makes it clear when you might want microservices: you're going through a period of hypergrowth and deployment is a bigger bottleneck than code. This made sense for DoorDash during covid, but that's a very unusual circumstance
I see a lot of value in spinning up microservices where the database is global across all services (and not inside the service) but I struggle more to see the value of separate core transactional databases for separate services unless/until the point where two separate parts of the organizations are almost two separate companies that cannot operate as a single org/single company. You lose data integrity, joining ability, one coherent state of the world, etc.
The main time I can see this making sense is when the data access patterns are so different in scale and frequency that they're optimizing for different things that cause resource contention, but even then, my question would become do you really need a separate instance of the same kind of DB inside the service, or do you need another global replica/a new instance of a new but different kind of DB (for example Clickhouse if you've been running Postgres and now need efficient OLAP on large columnar data).
Once you get to this scale, I can see the idea of cell-based architecture [1] making sense -- but even at this point, you're really looking at a multi-dimensionally sharded global persistence store where each cell is functionally isolated for a single slice of routing space. This makes me question the value of microservices with state bound to the service writ large and I can't really think of a good use case for it.
[1] https://docs.aws.amazon.com/wellarchitected/latest/reducing-...
> I see a lot of value in spinning up microservices where the database is global across all services (and not inside the service)
This issue with this is schema evolution. As a very simple example, let's say you have a User table, and many microservices accessing this table. Now you want to add an "IsDeleted" column to implement soft deletion; how do you do that? First you need to add the actual column to the database, then you need to go update every single service which queries that table and ensure that it's filtering out IsDeleted=True, deploy all those services, and only then can you actually start using the column. If you must update services in lockstep like this, you've built a distributed monolith, which is all of the complexity of microservices with none of the benefits.
A proper service-oriented way to deal with this is have a single service with control of the User table and expose a `GetUsers` API. This way, only one database and its associated service needs to be updated to support IsDeleted. Because of API stability guarantees--another important guarantee of good SoA--other services will continue to only get non-deleted users when using this API, without any updates on their end.
> You lose data integrity, joining ability, one coherent state of the world, etc.
You do lose this! And it's one of the tradeoffs, and why understanding your domain is so important for doing SoA well. For subsets of the domain where data integrity is important, it should all be in one database, and controlled by one service. For most domains, though, a lot of features don't have strict integrity requirements. As a concrete though slightly simplified example, I work with IoT time-series data, and one feature of our platform is using some ML algorithms to predict future values based on historical trends. The prediction calculation and storage of its results is done in a separate service, with the results being linked back via a "foreign key" to the device ID in the primary database. Now, if that device is deleted from the primary database, what happens? You have a bunch of orphaned rows in the prediction service's database. But how big of a deal is this actually? We never "walk back" from any individual prediction record to the device via the ID in the row; queries are always some variant of "give me the predictions for device ID 123". So the only real consequence is a bit of database bloat, which can be resolved via regularly scheduled orphan checking processes if it's a concern.
It's definitely a mindshift if you're used to a "everything in one RDBMS linked by foreign keys" strategy, but I've seen this successfully deployed at many companies (AWS, among others).
It's funny that we've now been having this conversation on HN for at least a decade.
I found a different benefit to micro services — AI understands them and context matters. Monolithic app confuse ai where micro services enables them to be far more effective.
I don’t want or need microservices.
I want just services.
The other problem is that very very few people actually know how to design a microservice based architecture. I've worked with half a dozen different teams who claim they're building microservices, but when you look at the system it's just a giant distributed monolith. Most of them are people who worked in legacy code bases, and while they like the idea of microservices, they can't let go of those design patterns. So they do the exact same thing but just out everything behind network calls. Drives me absolutely fucking nuts
Another good use case for a microservice - if you are going to have to change the compute size for your monolith just to accommodate the new functionality.
I had an architect bemoan the suggestion we use a microservice, until he had to begrudgingly back down when he was told that the function we were talking about (Running a CLIP model) would mean attaching a GPU to every task instance.
We've removed/merged most of the unnecessary services. The ones left have operational needs to stay separate.
The current hell is x years of undisciplined (in terms of perf and cost) new ORM code being deployed (SQLAlchemy). We do an insane number of read queries per second relative to our usage.
I honestly think the newish devs we have hired don't understand SQL at all. They seem to think of it as some arcane low level thing people used in the 80s/90s.
During a major site rewrite, one of my junior cohorts, suggested a monolithic re-entrant site... Easily tripled the TPS, and halved the response time.
I was stunned... He comes up with this stuff all the time. Thanks Matt.
We watched kernels go from monoliths to micro to hybrid.
And, now, SAAS is finally making the jump to the last position - hybrid/mini
Usually no
I don't want microservices!
What I want is a lightweight infrastructure for macro-services. I want something to handle the user and machine-to-machine authentication (and maybe authorization).
I don't WANT the usual K8s virtual network for that, just an easy-to-use module inside the service itself.
You should be able to spin up everything localy in a docker-compose container.
> What I want is a lightweight infrastructure for macro-services. I want something to handle the user and machine-to-machine authentication (and maybe authorization).
> I don't WANT the usual K8s virtual network for that, just an easy-to-use module inside the service itself.
K8s makes sense if you have a dedicated team (or atleast engineer) and if you really need need the advanced stuff (blue/green deployments, scaling, etc). Once it's properly setup it's actually a very pleasant platform.
If you don't need that Docker (or preferable Podman) is indeed the way to go. You can actually go quite far with a VPS or a dedicated server these day. By the time you outgrow the most expensive server you can (reasonable) buy you can probably afford the staff to roll out a "big boy" infrastructure.
I tried to use K8s several times, and I just can't make it work. It's fine as a deployment platform, but I just can't justify its complexity for local development.
We're using Docker/Podman with docker-compose for local development, and I can spin up our entire stack in seconds locally. I can attach a debugger to any component, or pull it out of the Docker and just run it inside my IDE. I even have an optional local Uptrace installation for OTEL observability testing.
My problem is that our deployment infrastructure is different. So I need to maintain two sets of descriptions of our services. I'd love a solution that would unify them, but so far nothing...
I wouldn't use K8s for local development unless you have some system where there is a dev cluster and you can route traffic for particular pod to your local workstation.
Docker Compose for local development is fine. If your K8s setup is crazy complex that you need to test it locally, please stop.
Tilt? Skaffold? configuration isn't free, but a debugger on a local k8 cluster that's at least somewhat representative of prod is pretty handy once you do.
I tried Tilt, but it's still too complicated. For example, we have a computer vision model that is a simple Python service. When developing on macOS, it's not possible to use GPUs inside containers, so we need to run it locally on the host.
It's trivial with my current setup, but not really possible with Tilt.
I know this pain, though we’re running nomad not k8s as our cluster control plane. But local devs are stuck with docker-compose, so 2 different configurations for running locally versus in the production environment.
Unless you need horizontal scalability or clustering, Compose + Terraform is all you need.
With Compose, you get proper n-tier application containerization with immutability. By adding an infrastructure-as-code tool such as Terraform to abstract your IT environment, you can deploy your application on-premises, in the cloud, or at a customer site with a single command.
For clustering needs, there’s Incus, and finally, Kubernetes for very fast scalability (<mn), massive deployments on large clusters, cloud offloading, and microservices.
Almost nobody truly needs the complexity of Kubernetes. The ROI simply isn’t there for the majority of use cases.
The one thing I would like to preserve from microservices is stuff about database table hygiene.
Large, shared database tables have been a huge issue in the last few jobs that I have had, and they are incredibly labor intensive to fix.
In my experience basically everything being good in software is downstream of good data modelling.
It's partly why I've realised more over time that learning computer science fundamentals actually ends up being super valuable.
I'm not talking about anything particularly deep either, just the very fundamentals you might come across in year one or two of a degree.
It sort of hooks back in over time as you discover that these people decades ago really got it and all you're really doing as a software engineer is rediscovering these lessons yourself, basically by thinking there's a better way, trying it, seeing it's not better, but noticing the fundamentals that are either being encouraged or violated and pulling just those back out into a simpler model.
I feel like that's mostly what's happened with the swing over into microservices and the swing back into monoliths, pulling some of the fundamentals encouraged by microservices back into monolith land but discarding all the other complexities that don't add anything.
I learned this early and as a result I'm the guy who's trying to clean up other people's crap while they ship features and get promoted
A databases is a global variable in disguise.
Why big orgs use microservices: makes teams focused on a clear problem domain
Why small orgs use microservices: makes it nearly physically impossible to do certain classes of dumb shit
I feel that if you have multiple sets of application logic that need to access the same data, there should be an internal API between them and the database that keeps that access to spec.
Only allow clients to execute stored procedures?