Skip to main content

Actors: distributed objects that hold state

Decorate a class instead of a function and you get an actor: a long-lived object on a specific worker that keeps its state between method calls.

@eugo.hpc.distribute
class Model:
def __init__(self, path):
self.model = load_model(path) # expensive, happens once

def predict(self, batch):
return self.model(batch)

model = Model.options(num_gpus=1).remote("s3://bucket/model.pt")

futures = [model.predict.remote(b) for b in batches]
results = eugo.hpc.get(futures)

When you need one​

The signal is expensive setup you need repeatedly.

Tasks are stateless: every call starts fresh. If loading a model takes 30 seconds and inference takes 50 ms, a thousand tasks spend eight hours loading and under a minute inferring.

An actor loads once, then answers a thousand calls.

Common cases:

  • A large model held in memory across many inference calls
  • A database or service connection with costly handshake
  • A cache built up over a run
  • Any resource where construction dominates use

When you do not​

Actors are not a general improvement over tasks. Prefer a task whenever setup is cheap.

An actor is a single object on a single worker, which means:

  • It is a serialization point. Calls to one actor queue. Ten thousand calls to one actor run sequentially, however large your cluster.
  • It occupies a worker for its lifetime, whether busy or not.
  • It can fail as a unit. Lose that worker and you lose the state.

A task, by contrast, runs anywhere and scales with the cluster.

The heuristic

If setup is cheap, use a task. Reach for an actor only when construction cost genuinely dominates.

Scaling actors: a pool​

One actor is a bottleneck. Several give you parallelism with amortized setup:

POOL_SIZE = 8
pool = [Model.options(num_gpus=1).remote(MODEL_PATH) for _ in range(POOL_SIZE)]

futures = [
pool[i % POOL_SIZE].predict.remote(batch)
for i, batch in enumerate(batches)
]
results = eugo.hpc.get(futures)

Eight loads instead of a thousand, and eight-way parallelism. Round-robin is fine when batches are similar in cost; with uneven work, hand batches to whichever actor is free rather than by index.

Actors for coordination​

Because an actor is a single owner of its state, it is a safe place for state that many tasks touch. The alternative is uncoordinated writers racing each other:

@eugo.hpc.distribute
class Progress:
def __init__(self):
self.done = 0
self.failed = 0

def record(self, ok):
if ok:
self.done += 1
else:
self.failed += 1

def summary(self):
return {"done": self.done, "failed": self.failed}

p = Progress.remote()
eugo.hpc.get([p.record.remote(ok) for ok in (True, True, False)])
print(eugo.hpc.get(p.summary.remote())) # {'done': 2, 'failed': 1}

Use this sparingly. Every task reporting to one actor makes that actor a bottleneck. That is fine for occasional progress updates, and wrong for per-record accounting.

Shutting one down​

An actor holds its worker until you release it. Long-running notebooks accumulate forgotten actors, and each is capacity you are paying for.

Release one when you are done with it:

eugo.hpc.kill(p)

Dropping your handle (del p) also releases the actor, but only once no other copy of the handle is left, including copies you passed to tasks.

Choosing between them​

TaskActor
State between callsNoneRetained
Runs onAny workerOne specific worker
Scales with clusterYesOnly via a pool
Setup cost paidEvery callOnce
Good forIndependent workExpensive, reusable state

Video: Actors for stateful work. The same material, with a transcript.