Skip to main content

Independent work

Units of work that can each produce their result without needing another unit’s output. A precondition for distributing anything.

The property that decides whether distribution helps at all. If each unit needs another unit’s result, the work is sequential no matter how many workers are available.

Distributed computeFundamentals

Related terms

  • Task: One unit of distributed work: a single call to a function decorated with the distribute decorator. Tasks are stateless, so each call starts fresh.
  • Partition: One independently processable chunk of a dataset. Partition size is the main lever on how well a workload parallelizes.
  • Shared state: Data that more than one task reads or writes. Shared mutable state is what turns a parallel run into a race condition.
  • Distributed computing: Splitting one job across several machines that work on it simultaneously. The core technique of HPC, and what the distribute decorator gives you in Python.