Skip to main content

Task

One unit of distributed work: a single call to a function decorated with the distribute decorator. Tasks are stateless, so each call starts fresh.

Stateless by design: each call starts fresh, receives copies of its arguments, and shares nothing with your session. When that model is wrong, because setup is expensive and needs reusing, an actor is the alternative.

Distributed computeSDK & API

Related terms

  • Actor: A distributed object that holds state across many calls. Created by applying the distribute decorator to a class rather than a function.
  • Future: A handle to a result that does not exist yet. Calling a distributed function returns a future immediately, while the work happens elsewhere.
  • distribute decorator: The `eugo.hpc` decorator that turns a function into a distributed task, or a class into an actor. A decorated call returns a future instead of a value.
  • Worker: A process on a compute node that executes tasks. Worker count, not node count, is what determines how many tasks run at once.
  • Distributed computing: Splitting one job across several machines that work on it simultaneously. The core technique of HPC, and what the distribute decorator gives you in Python.
  • Batching: Grouping several small units of work into one task so that scheduling overhead does not dominate the time spent doing useful work.
  • Fan-out: Launching many independent tasks at once, then waiting on all of their results together.
  • Serialization: Converting Python objects into bytes so they can be sent to another machine. Large task arguments are serialized and copied to every worker that needs them.
  • Shared state: Data that more than one task reads or writes. Shared mutable state is what turns a parallel run into a race condition.
  • Independent work: Units of work that can each produce their result without needing another unit’s output. A precondition for distributing anything.