Actors: distributed objects that hold state

Decorate a class instead of a function and you get an actor: a long-lived object on a specific worker that keeps its state between method calls.

@eugo.hpc.distribute
class Model:
    def __init__(self, path):
        self.model = load_model(path)      # expensive, happens once

    def predict(self, batch):
        return self.model(batch)

model = Model.options(num_gpus=1)("s3://bucket/model.pt")

futures = [model.predict(b) for b in batches]
results = eugo.hpc.get(futures)

When you need one

The signal is expensive setup you need repeatedly.

Tasks are stateless: every call starts fresh. If loading a model takes 30 seconds and inference takes 50 ms, a thousand tasks spend eight hours loading and under a minute inferring.

An actor loads once, then answers a thousand calls.

Common cases:

• A large model held in memory across many inference calls
• A database or service connection with costly handshake
• A cache built up over a run
• Any resource where construction dominates use

When you do not

Actors are not a general improvement over tasks. Prefer a task whenever setup is cheap.

An actor is a single object on a single worker, which means:

• It is a serialization point. Calls to one actor queue. Ten thousand calls to one actor run sequentially, however large your cluster.
• It occupies a worker for its lifetime, whether busy or not.
• It can fail as a unit. Lose that worker and you lose the state.

A task, by contrast, runs anywhere and scales with the cluster.

The heuristic

If setup is cheap, use a task. Reach for an actor only when construction cost genuinely dominates.

Scaling actors: a pool

One actor is a bottleneck. Several give you parallelism with amortized setup:

POOL_SIZE = 8
pool = [Model.options(num_gpus=1)(MODEL_PATH) for _ in range(POOL_SIZE)]

futures = [
    pool[i % POOL_SIZE].predict(batch)
    for i, batch in enumerate(batches)
]
results = eugo.hpc.get(futures)

Eight loads instead of a thousand, and eight-way parallelism. Round-robin is fine when batches are similar in cost; with uneven work, hand batches to whichever actor is free rather than by index.

Actors for coordination

Because an actor is a single owner of its state, it is a safe place for state that many tasks touch. The alternative is uncoordinated writers racing each other:

@eugo.hpc.distribute
class Progress:
    def __init__(self):
        self.done = 0
        self.failed = 0

    def record(self, ok):
        if ok:
            self.done += 1
        else:
            self.failed += 1

    def summary(self):
        return {"done": self.done, "failed": self.failed}

Use this sparingly. Every task reporting to one actor makes that actor a bottleneck. That is fine for occasional progress updates, and wrong for per-record accounting.

Shutting one down

An actor holds its worker until you release it. Long-running notebooks accumulate forgotten actors, and each is capacity you are paying for.

Choosing between them

	Task	Actor
State between calls	None	Retained
Runs on	Any worker	One specific worker
Scales with cluster	Yes	Only via a pool
Setup cost paid	Every call	Once
Good for	Independent work	Expensive, reusable state

---

Video: Actors for stateful work. The same material, with a transcript.