Skip to main content

Parallel computing

Many calculations at once rather than one after another. Two scales on Eugo: across machines in a cluster, and inside a CPU core through vectorization.

The two scales are worth keeping separate, because they are reached differently. Spreading work across machines is something you ask for, with the distribute decorator. Parallelism inside a core is something the runtime applies to your code without being asked, which is why the same loop can run faster after a rewrite that changed no logic.

Distributed computePerformance & optimization

Related terms

  • Distributed computing: Splitting one job across several machines that work on it simultaneously. The core technique of HPC, and what the distribute decorator gives you in Python.
  • Vectorization: Rewriting operations so a single instruction processes several data elements at once, using the processor’s wide registers.
  • High-performance computing (HPC): Running work across many machines at once so it finishes far sooner. The technique behind weather models, genomics, and large-scale simulation.