You have Godot wired up to talk to FoundationDB? That's incredibly cool, curious about your use case.
Godot runs using our container runtime on Rivet. The FDB API for containers is currently undocumented. I'd be happy to share early access on Discord – https://rivet.gg/discord
Games running on Rivet usually have a hybrid of actors that handle persistent storage (e.g. a matchmaker) and the game server itself.
I aim to build a collaborative VR world editor in Godot using mvsqlite for shared scene persistence.
I was salvaging someone else's toy project. The main problem with my research was write performance in SQLite on FDB. I didn't have the capacity to integrate Hctree, which was in the pipeline.
For ages I've been hoping someone will clone Durable Objects and here you are! I'm going to check this out.
Are there any gotchas or differences from Durable Objects behaviour when using Rivet Actors? There seems to be quite a lot of subtlety in some of how they handle e.g. IO, so I assume you wouldn't be compatible with this directly?
> Here's where Rivet's architecture gets fun – we don't rely on a traditional orchestrator like Kubernetes or Nomad for our runtime. Instead, our orchestrator is powered by an in-house actor-like workflow engine – similar to how FoundationDB is powered by their own actor library (Flow [4]) internally. It lets us reliably & efficiently build complex logic – like our orchestrator – that would normally be incredibly difficult to build correctly. For example, here's the logic that powers Rivet Actors themselves with complex mechanisms like retry upgrades, retry backoffs, and draining [2].
It is a bit unclear for me, do you use actors themself to develop Rivet Actors, or it is another actor-like workflow engine, not the final product?
(I would be super happy to read an article, which explains architecture, main blocks of the system, gives some an example)
How is this different from the Durable Promises idea by ResonateHQ?
It seams to me a bitter easer to get started with as a small standalone project.
Would you mind comparing this solutions to that?
I skimmed a bit about Durable Promises, and if I understand correctly, they’re similar to workflow tools like Temporal but come with high-quality language bindings that let you write workflow steps using async/await.
We built something almost identical in Rust to let us use async/await for long-running & failure-prone workflows. This powers almost everything we do at Rivet. We have a technical writeup coming later, but here's a couple links of interest:
Rivet Actors can start, stop, or crash at any time and still continue functioning, much like Durable Promises.
However, their scope differs: Rivet Actors are broader and designed for anything stateful & realtime, while Durable Promises seem focused on workflows with complex control flow.
Rivet Actors can (and likely will) support workflow-like applications in the future, since state management and rescheduling are already built-in.
What kinds of web sites need this level of complexity? They must be doing something really hard and tightly coordinated, such as collaborative editing, right?
Does this have enough performance to solve the server cost problem of Improbable's metaverse back end? That's a remote object system too expensive to run 24/7.
> What kinds of web sites need this level of complexity?
The most common use cases are high contention & strong consistency workloads where using an OLTP database doesn't work well. A few examples:
- Collaborative docs
- Local-first & CRDT sync (e.g. Yjs)
- Stateful multiplayer games, often async multiplayer (e.g. chess)
- Agentic workflows with frequent memory updates
- Chat
- Scheduling software
- Rate limiting
Actors can also be used for stateless realtime workloads that benefit from edge computing, similar to what you would use existing serverless edge functions for.
The actor model is widely used in larger companies. The best-known example is how WhatsApp used Erlang to scale to 900 M users with 50 devs [2]. It's also widely used at Discord, LinkedIn, Microsoft, etc. Take a look at Erlang/Elixir, Akka, and Orleans for prior art.
> Does this have enough performance to solve the server cost problem of Improbable's metaverse back end?
Actors usually have a single responsibility (e.g. document or chat room) while Improbable focused on building expansive worlds without clear boundaries between entities. Our docs include tips on scoping the responsibility of actors [1].
The actor model encourages actor-actor communication, sort of similar how entities in the ECS model communicate in games.
> That's a remote object system too expensive to run 24/7.
Actors automatically go to sleep when unused. They provide the benefit of realtime workloads without the cost of keep idle compute sitting around.
For example – I'm a big proponent of local-first apps, but even those require a sync server. You can easily implement one in under 100 lines of code with actors.
Haha, I’m with you there. I was considering calling that out in the post.
I dearly love Erlang & co., but its learning curve is way too steep for most developers today. Our goal is to bring the benefits of Erlang/Akka/Orleans/etc. to more developers by:
(a) supporting mainstream languages, and
(b) lowering the technical and conceptual barriers to entry.
Out of curiosity, what's the sort of consistency (in the sense of linearizable etc.,.) provided here? If an RPC is handled by some actor, is it guaranteed to have seen all previously accepted messages that completed up to that point?
If so, how does that work?
Edit: I guess this sort of comes by forcing everything to round trip through foundationdb for state? As long as the actor reads the state from foundationdb before handling the response, and writes to foundationdb before sending a reply, it should guarantee the same sort of consistency you'd get from foundationdb?
What about calls to other actors? If I make a potentially state changing RPC call to some other actor as a part of handling an RPC, do those commit together, or is it possible for the other actor to commit without me?
We document as many design decisions like this as we can. Here’s a related bit on serial vs. parallel RPC/message handling [1].
> I guess this sort of comes by forcing everything to round-trip through FoundationDB for state?
If you’re using the KV API directly, this is correct.
Actors also have a `this._state` property, which is automatically written to FDB after each RPC call if modified [2]. This allows developers to rapidly prototype by writing vanilla JS code like `this._state.count += 1` without having to worry about writing state and its associated edge cases.
> What about calls to other actors? If I make a potentially state-changing RPC call to some other actor as part of handling an RPC, do those commit together, or is it possible for the other actor to commit without me?
Not at the moment. You’d need to use a 2-phase commit for that.
It might be interesting if there were a way to represent a part of a transaction to commit as a sort of effect of an RPC. I don't know the details, but I'm aware of a non-public system that does a similar sort of thing to coordinate a distributed transaction between many different RPC services that are otherwise opaque.
E.g., you make a bunch of RPC calls, some part of the framework is tracking the read set and writes that need to be made, and then when the root of the RPC is ready to commit it can all atomically commit or fail (if there was a transaction conflict or something).
If I wanted a Rivet integration with Godot Engine and/or Elixir, how would I do it for shared durable objects in a virtual reality environment?
I have code for Godot Engine Foundationdb access, but I'm cutting too many encapsulation layers.
You have Godot wired up to talk to FoundationDB? That's incredibly cool, curious about your use case.
Godot runs using our container runtime on Rivet. The FDB API for containers is currently undocumented. I'd be happy to share early access on Discord – https://rivet.gg/discord
Games running on Rivet usually have a hybrid of actors that handle persistent storage (e.g. a matchmaker) and the game server itself.
I aim to build a collaborative VR world editor in Godot using mvsqlite for shared scene persistence.
I was salvaging someone else's toy project. The main problem with my research was write performance in SQLite on FDB. I didn't have the capacity to integrate Hctree, which was in the pipeline.
* https://github.com/V-Sekai/godot-mvsqlite
* https://github.com/V-Sekai/mvsqlite
* https://github.com/V-Sekai/elixir-mvsqlite
* https://sqlite.org/hctree/doc/hctree/doc/hctree/index.html
Let's connect on discord.
For ages I've been hoping someone will clone Durable Objects and here you are! I'm going to check this out.
Are there any gotchas or differences from Durable Objects behaviour when using Rivet Actors? There seems to be quite a lot of subtlety in some of how they handle e.g. IO, so I assume you wouldn't be compatible with this directly?
https://blog.cloudflare.com/durable-objects-easy-fast-correc...
> Here's where Rivet's architecture gets fun – we don't rely on a traditional orchestrator like Kubernetes or Nomad for our runtime. Instead, our orchestrator is powered by an in-house actor-like workflow engine – similar to how FoundationDB is powered by their own actor library (Flow [4]) internally. It lets us reliably & efficiently build complex logic – like our orchestrator – that would normally be incredibly difficult to build correctly. For example, here's the logic that powers Rivet Actors themselves with complex mechanisms like retry upgrades, retry backoffs, and draining [2].
It is a bit unclear for me, do you use actors themself to develop Rivet Actors, or it is another actor-like workflow engine, not the final product?
(I would be super happy to read an article, which explains architecture, main blocks of the system, gives some an example)
How is this different from the Durable Promises idea by ResonateHQ? It seams to me a bitter easer to get started with as a small standalone project. Would you mind comparing this solutions to that?
I skimmed a bit about Durable Promises, and if I understand correctly, they’re similar to workflow tools like Temporal but come with high-quality language bindings that let you write workflow steps using async/await.
We built something almost identical in Rust to let us use async/await for long-running & failure-prone workflows. This powers almost everything we do at Rivet. We have a technical writeup coming later, but here's a couple links of interest:
- Rough overview: https://github.com/rivet-gg/rivet/blob/58b073a7cae20adcf0fa3...
- Example async/await-heavy workflow: https://github.com/rivet-gg/rivet/blob/58b073a7cae20adcf0fa3...
- Example actor-like workflow: https://github.com/rivet-gg/rivet/blob/58b073a7cae20adcf0fa3...
Rivet Actors can start, stop, or crash at any time and still continue functioning, much like Durable Promises.
However, their scope differs: Rivet Actors are broader and designed for anything stateful & realtime, while Durable Promises seem focused on workflows with complex control flow.
Rivet Actors can (and likely will) support workflow-like applications in the future, since state management and rescheduling are already built-in.
For a deeper dive, Temporal has a writeup comparing actors and workflows: https://temporal.io/blog/workflows-as-actors-is-it-really-po...
Related discussion in 2023 https://news.ycombinator.com/item?id=37188659
Yep! Same repo, lots of changes. Since we were on HN in 2023, we've:
- Rewrote Nomad from scratch with our own in-house workflow engine
- Launched Rivet Actors and isolate runtime
- Overhauled our OSS experience to make self-hosting easier
- Rewrote frontend from scratch with shadcn to be like the cool kids
- Replaced Nebula with mTLS
- Completely stripped out Redis for non-caching use cases
We've also opened up the platform to non-gaming use cases.
I've added that link above. Thanks!
What kinds of web sites need this level of complexity? They must be doing something really hard and tightly coordinated, such as collaborative editing, right?
Does this have enough performance to solve the server cost problem of Improbable's metaverse back end? That's a remote object system too expensive to run 24/7.
Hey!
> What kinds of web sites need this level of complexity?
The most common use cases are high contention & strong consistency workloads where using an OLTP database doesn't work well. A few examples:
- Collaborative docs
- Local-first & CRDT sync (e.g. Yjs)
- Stateful multiplayer games, often async multiplayer (e.g. chess)
- Agentic workflows with frequent memory updates
- Chat
- Scheduling software
- Rate limiting
Actors can also be used for stateless realtime workloads that benefit from edge computing, similar to what you would use existing serverless edge functions for.
The actor model is widely used in larger companies. The best-known example is how WhatsApp used Erlang to scale to 900 M users with 50 devs [2]. It's also widely used at Discord, LinkedIn, Microsoft, etc. Take a look at Erlang/Elixir, Akka, and Orleans for prior art.
> Does this have enough performance to solve the server cost problem of Improbable's metaverse back end?
Actors usually have a single responsibility (e.g. document or chat room) while Improbable focused on building expansive worlds without clear boundaries between entities. Our docs include tips on scoping the responsibility of actors [1].
The actor model encourages actor-actor communication, sort of similar how entities in the ECS model communicate in games.
> That's a remote object system too expensive to run 24/7.
Actors automatically go to sleep when unused. They provide the benefit of realtime workloads without the cost of keep idle compute sitting around.
[1] https://rivet.gg/docs/scaling#tips-for-architecting-actors-f...
[2] https://www.wired.com/2015/09/whatsapp-serves-900-million-us....
If you have a really tiny bit of compute that must happen on the edge, I guess it makes sense.
For example – I'm a big proponent of local-first apps, but even those require a sync server. You can easily implement one in under 100 lines of code with actors.
Here’s my once a month comment that I should probably put on autopilot:
“at least half of show HN posts are just reimplementations of what already exits in Erlang/Elixir”.
The actor model existed before Erlang, and Erlang doesn't have some kind of a monopoly on it. Akka is pretty good too.
Same as every OOP isn't "something that exists already in Java"
Haha, I’m with you there. I was considering calling that out in the post.
I dearly love Erlang & co., but its learning curve is way too steep for most developers today. Our goal is to bring the benefits of Erlang/Akka/Orleans/etc. to more developers by:
(a) supporting mainstream languages, and
(b) lowering the technical and conceptual barriers to entry.
Erlang has its quirks but Elixir at least is an order of magnitude easier to learn than Rust or Java.
Meanwhile, I'm a rust developer who is struggling to learn elixir :D
The key here is Typescript.
Out of curiosity, what's the sort of consistency (in the sense of linearizable etc.,.) provided here? If an RPC is handled by some actor, is it guaranteed to have seen all previously accepted messages that completed up to that point?
If so, how does that work?
Edit: I guess this sort of comes by forcing everything to round trip through foundationdb for state? As long as the actor reads the state from foundationdb before handling the response, and writes to foundationdb before sending a reply, it should guarantee the same sort of consistency you'd get from foundationdb?
What about calls to other actors? If I make a potentially state changing RPC call to some other actor as a part of handling an RPC, do those commit together, or is it possible for the other actor to commit without me?
We document as many design decisions like this as we can. Here’s a related bit on serial vs. parallel RPC/message handling [1].
> I guess this sort of comes by forcing everything to round-trip through FoundationDB for state?
If you’re using the KV API directly, this is correct.
Actors also have a `this._state` property, which is automatically written to FDB after each RPC call if modified [2]. This allows developers to rapidly prototype by writing vanilla JS code like `this._state.count += 1` without having to worry about writing state and its associated edge cases.
> What about calls to other actors? If I make a potentially state-changing RPC call to some other actor as part of handling an RPC, do those commit together, or is it possible for the other actor to commit without me?
Not at the moment. You’d need to use a 2-phase commit for that.
[1] https://rivet.gg/docs/internals/design-decisions#parallel-rp...
[2] https://rivet.gg/docs/state#state-saves
Got it, thanks for the reply!
It might be interesting if there were a way to represent a part of a transaction to commit as a sort of effect of an RPC. I don't know the details, but I'm aware of a non-public system that does a similar sort of thing to coordinate a distributed transaction between many different RPC services that are otherwise opaque.
E.g., you make a bunch of RPC calls, some part of the framework is tracking the read set and writes that need to be made, and then when the root of the RPC is ready to commit it can all atomically commit or fail (if there was a transaction conflict or something).
> written to FDB after each RPC call
What does "after" mean in this case? Is the state change durable in FDB before the client sees the result of the RPC call?
Looks interesting. I am wondering if I could get this to run with F5 XC