Learning the actor model through a real case

This post is based on my talk “Aprendiendo sobre el modelo de actores a través de un caso real” (in Spanish), which I gave at Commit Conf. The recording is on Sirviendo Código.

In the village where I live, years ago, the only way to Madrid was the morning bus, and the only way back was the afternoon one. So if you drove back from the city, you would often stop at the bus stop and offer someone a ride. A second cousin of mine, who worked at Telefónica, picked up an old man that way. They got talking, and the old man asked where he worked. “At Telefónica.” “Ah, so you put up poles or you lay cables.”

Put up poles or lay cables: for that man, those were the only two jobs at a phone company.

For that man there was no third option. In software we often do the same thing. Our toolbox could be huge, but we limit ourselves to a few tools, and conference agendas show it: one year everything is Kubernetes, the next one everything is microservices, and this year it is AI.

This post is about adding one more tool to the box: the actor model. Not through a Netflix-scale project you will never get to apply, but through a small, real problem where the actor model fit, and where I used it with as little effort as I could while getting as much out of it as possible.

A bit of history

The actor model comes from a 1973 paper by Carl Hewitt, Peter Bishop and Richard Steiger: A Universal Modular ACTOR Formalism for Artificial Intelligence. It proposes a modular architecture for artificial intelligence based on a single kind of object, the actor. Read it today and it sounds a lot like what we now say about agents. We keep going in circles.

Everybody remembers Hewitt and forgets the other two authors. Hewitt died in 2022, at 77. So we are talking about something very old, and that context matters. One of the selling points of the paper is that the architecture “does not have or need goto, interrupt, or semaphore primitives”. Two paragraphs later it tells you that “Monotheism is the Answer”. It also quotes Macbeth and Alice in Wonderland. Papers used to be more fun to read.

In later explanations Hewitt describes the model as a mathematical theory that treats actors as the universal primitives of computation, with a bold hypothesis: anything that can physically be computed can be implemented with actors. And he adds that message passing using types is the foundation of communication in the system.

That sounds familiar, because at the same time Alan Kay was saying that, for him, object orientation means only messaging, local retention and protection of state, and extreme late binding. Those were years when Lisp, Simula and Smalltalk all influenced each other, and the actor model drew on all of them. If you put an implementation of the actor model next to Kay’s definition of object orientation, you can say they match. But there are differences too. Kay preferred dynamic typing, while Hewitt puts types at the foundation. Similar ideas, different schools, and that can confuse us.

Two primitives

The actor model is so simple that you could draw it with two boxes and a line. It rests on two primitives: messages and actors.

The actor model in one picture. It is the only AI-generated image in the talk, and it has one actor more than it needs.

A message is a unit of communication that carries information between actors. It is the envelope with the letter inside, or the note you passed to a classmate as a teenager. It has a few characteristics:

• It is immutable, or at least it is supposed to be.
• It is asynchronous. More precisely, messages are what enables asynchrony.
• It is a container for data or for instructions, much like the messages in a system with queues or in CQRS.
• It decouples actors. All I depend on is the contract of the message, its schema and, depending on the implementation, the protocol. I don’t need to know where the other actor lives.

An actor is a unit of computation that can do three things:

• Create other actors.
• Send messages to other actors.
• Process the messages it receives, one at a time.

That last point is where the value of the model lies. The actor model sells itself as an easy way to understand, reason about and diagnose concurrency. And yet it starts by saying that each actor processes its messages sequentially. So where is the concurrency? That is the shift in thinking the model asks for, and we will get there.

How it works

Each actor has a mailbox. Some implementations call it an inbox, and some don’t name it at all, but the idea is the same: messages arrive in the actor’s mailbox, and the actor processes them in order. In which order exactly depends on the implementation, but for now assume it is the order in which they were sent.

Say the actor system sends a Create message to the /user actor. Processing it, /user creates a new actor, /user/123, which from then on handles the messages for that user: change the contact address, reset the password, whatever your business needs.

The paths are an Akka detail I will come back to. The general idea doesn’t depend on them.

Concurrency comes from having many actors

If each actor processes its messages one by one, concurrency comes from having many actors. Processing is sequential inside an actor and parallel between actors.

Back to the example. If I want creating users to go through one line and changing profiles to go through another, I split them into different actors. Creating a user is a message to /user. Resetting the password of user 123 is a message to /user/123. And if profiles change often, I give them their own actor, /user/123/profile. Each actor processes its messages sequentially and manages its own state, and I get parallelism because I broke the work down into several actors.

Actors also form a hierarchy. Every actor is created by another one, so the system ends up as a tree: /user and /order under the root, /user/123 under /user, /user/123/profile under that. We will see why the hierarchy matters when we get to the real case.

The original paper doesn’t tie any of this to an implementation. It even mentions building actors directly in hardware, as a circuit. That openness shaped how the model evolved.

Isn’t this just DDD entities with CQRS?

If you have been to talks about DDD and CQRS, the heavy kind with commands and Kafka, this may ring a bell: I send a command to an entity, the entity protects its invariants. Some parts of the solution are shared, yes. But the difference, looking at the model itself and not at any implementation, is how it approaches concurrency.

The actor model and DDD with CQRS, looking at each other.

In DDD you are told to protect the invariants in your aggregates to keep the system consistent. The actor model makes it much simpler: choose what must run in sequence and what can run in parallel. Do you need parallelism? Create more actors. Is there something you want to keep under control at all times, like generating invoices in order with a well-controlled state? Create an actor for it, and every message it receives will be processed in sequence.

With actors you don’t deal with race conditions, semaphores or locks. With other architectural patterns you end up facing them, because it is inherent to how they work.

Many models, many implementations

When I first opened the Akka documentation it talked about persistence, streams, remoting, clustering, scheduling and much more. All of that exists in frameworks like Akka, but it belongs to the implementation, not to the model. Distributed systems today are very different from the ones people imagined more than fifty years ago, and the paper itself refused to commit to any implementation. It is like CQRS: CQRS doesn’t require separate storage for reads and writes, or a message broker. The actor model doesn’t require any of those extras either.

A short paper from 2016 shows this well: 43 Years of Actors: A Taxonomy of Actor Models and Their Key Properties, by Joeri De Koster, Tom Van Cutsem and Wolfgang De Meuter. It compares implementations by how they process messages, how they receive them, how they handle state changes and what their execution environment looks like, and it describes different schools, the same way there are different schools of object orientation: classic actors like Akka, processes like Erlang, active objects like Orleans, and communicating event loops like E.

Take message order as an example. FIFO sounds familiar, but the paper also uses terms like out-of-order execution or E-ORDER, which describe at which level order is guaranteed. Akka guarantees that the messages one actor sends to another arrive in the order they were sent, and nothing more: if you need to coordinate the order of messages across several actors, it won’t do it for you. E, with its E-ORDER, gives you stronger guarantees on causality between messages.

There is no single flavor of the actor model. Even among today’s frameworks, basic concepts like the hierarchy between actors or how you reach an actor change from one to another:

	Akka / Akka.NET	Erlang / Elixir	Orleans	Pykka	Pony
Hierarchy between actors	Yes	Yes	No	Limited	Optional
Supervision	Built in	Very strong	Not explicit	Basic	No
Access by name or path	Yes (ActorSelection)	Yes (registered name)	Yes (logical ID)	No	No

The details of each cell matter less than the point: your framework decides what you can do and how you will work. If you need a distributed system, for example, virtual actors like the ones in Orleans, which you don’t reach by path, may make your life easier.

Show me the code

The examples use C# with Akka.NET. In Java with Akka it would look almost the same, and in Kotlin it would look nicer than in C#, I admit it. I will start with the kind of examples ChatGPT would give you, and then move to a summarized version of a real project.

An actor

An actor that prints whatever it receives:

public class PingActor : ReceiveActor
{
    public PingActor()
    {
        Receive<string>(msg =>
        {
            Console.WriteLine($"[PingActor] Received: {msg}");
        });
    }
}

A class that inherits from ReceiveActor and declares which messages it can receive, here a string. Very strongly typed, I know. That is all an actor is.

To use it:

using var system = ActorSystem.Create("PingSystem");
IActorRef pingActor = system.ActorOf<PingActor>("pingActor");
pingActor.Tell("ping");

The first line starts the actor system, something you do once, when the application starts. The second one creates an actor of type PingActor, and its argument is the actor’s name, which becomes part of its path. This is where Akka confuses newcomers: every actor you create lives under /user, so this one is /user/pingActor. Next to /user there is /system, for the actors Akka creates for itself, and /temp, for temporary actors that will show up in a moment.

The third line sends a message with Tell. The actor processes it asynchronously, the late binding we mentioned when talking about object orientation. Every message sent to this actor is processed in sequence, and the actor can manage its state from the inside without any lock.

Request and response

Usually we want to ask for something and get an answer. And since the whole world can be modeled with actors and messages, the actor model says: do it with actors and messages. A ClientActor sends ping to the PingActor, and the PingActor replies with pong to whoever sent the message:

public class PingActor : ReceiveActor
{
    public PingActor()
    {
        Receive<string>(msg =>
        {
            Sender.Tell("pong");
        });
    }
}

public class ClientActor : ReceiveActor
{
    public ClientActor(IActorRef pingActor)
    {
        Receive<string>(msg =>
        {
            Console.WriteLine($"[ClientActor] Received reply: {msg}");
        });
        pingActor.Tell("ping", Self);
    }
}

using var system = ActorSystem.Create("PingSystem");
var pingActor = system.ActorOf<PingActor>("pingActor");
var clientActor = system.ActorOf(
    Props.Create(() => new ClientActor(pingActor)),
    "clientActor");

The client receives a reference to the PingActor so that it can send it messages, and when it does, it says who it is with Self. That is how the PingActor knows where to send the answer.

You rarely write it like that. Akka.NET gives you Ask:

using var system = ActorSystem.Create("PingSystem");
var pingActor = system.ActorOf<PingActor>("pingActor");
var response = await pingActor.Ask<string>("ping");
Console.WriteLine($"[Main] Received from actor: {response}");

The PingActor doesn’t change: it still replies to Sender. What disappears is the ClientActor. Behind the scenes, Akka creates a temporary actor under /temp that plays that role. Underneath, it is still actors talking to each other.

Timers

Akka also has timers. Not every implementation of the actor model has them, but Akka does, and they let you say: send this message to this actor every 10 seconds.

public class PingActor : ReceiveActor, IWithTimers
{
    private const string PingTimerKey = "ping-timer";
    public ITimerScheduler Timers { get; set; }

    public PingActor()
    {
        Receive<string>(msg =>
        {
            Console.WriteLine($"[{DateTime.Now:T}] [PingActor] Received: {msg}");
        });
    }

    protected override void PreStart()
    {
        Timers.StartPeriodicTimer(
            key: PingTimerKey,
            msg: "ping",
            interval: TimeSpan.FromSeconds(10)
        );
    }
}

The actor implements IWithTimers, and in PreStart it starts a periodic timer that sends it ping every 10 seconds. The actor schedules itself. With that you have scheduled tasks without Quartz or anything similar.

The real case: monitoring

I work in a sector where we sometimes have to deal with musty environments. By musty I mean old-school on-premise: no Kubernetes cluster anywhere in sight.

When we talk about monitoring, the first thing that comes to mind is Grafana. But if all I want is to know whether ten services are up, Grafana is a cannon to kill a fly, unless you already have it. And on-premise there is more: I have to ask the owner of the infrastructure for permission, and the answer will be no, because it means a new server and securing a platform they don’t control. Prometheus? They don’t work with Prometheus.

On-premise, things are sometimes closer to the maker world, more homemade. Our product, installed on-premise, has lighter requirements than the recommended setup for a Kafka cluster. If I need event streams, everybody will tell me to set up Kafka, and that would double the infrastructure costs.

In the maker world there is a monitoring tool called Uptime Kuma. It has very nice things, but it is written in Node, which means one more new dependency for the client’s security office to look at, and it has no API, so configuring it from code is out. It gets you out of trouble until you need those details.

What we can do is add new applications written in our own stack, .NET: a new website, a new API, using the persistence we already have. And what did I need? Monitoring of basic services (HTTP, FTP, ping and so on), basic alerts, and a persisted history, so that after a restart I can still see how the last month went.

That became Uptime Bonney. It is a list of services with a traffic light for each one, plus availability percentiles and response times. Uglier than the alternatives, but it covers our use case, I don’t have to argue with the client, and it is easy to use. Creating a monitor is a form, or a bit of JSON, with a Test button that checks it on the spot.

Uptime Bonney running in a development environment, with simulated monitors that fail on purpose.

Creating or editing a monitor. The Test button checks it on the spot.

The actor model doesn’t run the whole thing. The CRUD of monitors is a plain API with a service and a repository, because there is no parallelism or shared state to protect there. Actors only run the monitors.

The first design, and why it doesn’t work

If you come from the world of entities, the first idea goes like this: in the same way I would create a monitors endpoint or a monitors service, I create a monitors actor, and with Akka.NET timers I ask it to check each service every few seconds.

That is a mistake, and you can probably see why: concurrency. With 300 monitors, each one asking for a check every second, everything goes through one mailbox, and the actor processes it sequentially. If a check takes a second because a server is slow to answer, the messages pile up. Make the same actor persist each result, and every check also waits for the database. The system saturates.

The second design: a hierarchy

The fix is the hierarchy. There is a supervisor, a concept you meet a lot in the actor model. The supervisor only handles the basic management: creating monitors and removing them. Nothing else. Each monitor is a child actor, and when it is created it starts a timer that sends it a message to check its service.

Next to them there is a status store, an actor whose only job is to save results. It saves them sequentially, and that doesn’t matter: storing data no longer gets in the way of running monitors. The only cost is a small delay in persistence.

The monitor actor

The messages are typed now, instead of the strings from the earlier examples. The monitor declares the four messages it understands:

public class MonitorActor : ReceiveActor, IWithTimers
{
    public ITimerScheduler? Timers { get; set; }

    public MonitorActor(HttpClient httpClient)
    {
        Receive<TestMonitor>(message => { /* your business here */ });
        Receive<VerifyMonitor>(message => { /* your business here */ });
        Receive<UnScheduleMonitor>(message => { /* your business here */ });
        Receive<ScheduleMonitor>(message => { /* your business here */ });
    }
}

Each handler does something very small. TestMonitor backs the Test button, where I do want request and response: it checks the monitor and replies to whoever asked. VerifyMonitor is the regular, asynchronous check: it checks the monitor, finds the status store by its path and tells it to register the result. Then it is done. Saving is the store’s problem.

Receive<TestMonitor>(message =>
{
    var status = CheckMonitorStatus(httpClient, message.Monitor);
    Sender.Tell(status);
});
Receive<VerifyMonitor>(message =>
{
    var status = CheckMonitorStatus(httpClient, message.Monitor);
    var actorPath = "/user/monitor-status-store";
    var actorSelection = Context.System.ActorSelection(actorPath);
    actorSelection.Tell(new RegisterMonitorStatus(status));
});

Scheduling and unscheduling are the timers again. Scheduling a monitor cancels its previous timer if it had one and, if the monitor is enabled, starts a periodic timer that sends it VerifyMonitor at the monitor’s interval:

Receive<ScheduleMonitor>(message =>
{
    if (Timers != null && Timers.ActiveTimers.Contains(message.Monitor.Id))
    {
        Timers.Cancel(message.Monitor.Id);
    }
    if (Timers == null || !message.Monitor.Enabled) return;
    Timers.StartPeriodicTimer(message.Monitor.Id,
                              new VerifyMonitor(message.Monitor),
                              TimeSpan.FromSeconds(message.Monitor.Interval));
});
Receive<UnScheduleMonitor>(message =>
{
    if (Timers == null || !Timers.ActiveTimers.Contains(message.Id)) return;
    Timers.Cancel(message.Id);
});

Persistence in batches

If I had 1,000 monitors and wrote to the database after every check, I would saturate the database. So the status store doesn’t write each result when it arrives. It adds it to a batch in memory, which is fast and safe because the state belongs to the actor and only the actor touches it. Then a timer tells it, every 10 seconds, to save the batch for real with a bulk insert:

public class MonitorStatusStoreActor : ReceiveActor, IWithTimers
{
    public ITimerScheduler? Timers { get; set; }

    public MonitorStatusStoreActor(IMonitorStatusStore store)
    {
        var nextBatch = store.GetEmptyStatusDataTable();
        Receive<RegisterMonitorStatus>(message =>
        {
            store.AddStatusToDataTable(nextBatch, message.Status);
        });
        Receive<SaveMonitorStatusBatch>(_ =>
        {
            store.SaveStatusBatch(nextBatch).GetAwaiter().GetResult();
            nextBatch.Clear();
        });
    }

    protected override void PreStart()
    {
        base.PreStart();
        Timers?.StartPeriodicTimer("batch-store-timer",
                                   new SaveMonitorStatusBatch(),
                                   TimeSpan.FromSeconds(10));
    }
}

The timer’s message goes through the same mailbox as the results, so a batch is never saved while a result is being added to it. Parallelism where I want it, sequence where I need it, and no locks.

When to use actors

Not for everything, no way. As I said, the CRUD of monitors in this project is a plain API with a service and a repository, because there is nothing there that needs it.

The actor model shines when you need to manage state with a high level of parallelism, but with sequential processing per entity. Think of invoicing: I want everything inside one invoice book to happen in sequence, but I have 200 invoice books, and those I want in parallel. With actors there are no race conditions to deal with. You only need to make one mental switch: my unit of parallelism is having many actors, and inside each actor everything is sequential.

Personally, I would only use it in specific corners of a system, unless you work in Elixir, where you use it all the time. In .NET, Java, Kotlin or PHP, I would consider it when managing parallelism starts to hurt. And the cost is small: in Akka and Akka.NET an actor takes around 300 bytes of memory. If you can’t afford 300 bytes, and you work with a language that runs on a virtual machine, maybe you are in the wrong job.

Distributing actors

Running actors across several machines depends on the framework. Akka has remoting and clustering, and Orleans is distributed from the start. You send a message to an actor, and the framework finds out where that actor lives and takes care of its whole life cycle. Other implementations only work in a single process. I didn’t need any of that here: if I tell my client I am going to set up a cluster for monitoring, they will ask me what I am playing at.

What about Erlang? I am a hobbyist there, not an expert. In Erlang you work with actors all the time, without choosing, and someone coming from Erlang will tell you that theirs are native actors and yours are wannabe actors. You will find differences in philosophy, especially in how failures and supervision are handled. I can’t tell you which one is better, because that depends on the use case. For the problems I usually face, I prefer a framework I can bring into one corner of the system when I need it.

Other topics to explore

There is a lot more I left out: supervision strategies and how to handle errors, how to test actors, and how to observe them in production. Each one deserves its own post.