# Workload readiness checklist

Verify a workload is worth distributing before you spend cluster time on it.

Source: https://university.eugo.io/resources/workload-readiness-checklist

---

- Kind: Checklist
- Topics: distributed-compute, performance, fundamentals

## Understand the current cost

- **Time a single unit of work** — Measure how long one partition, file, or record batch takes on one machine. Every sizing decision downstream depends on this number, and guessing it wrong is the most common reason a cluster underdelivers.
- **Count the independent units** — Establish how many units exist. Serial time is roughly units multiplied by per-unit time, and that product is the ceiling on what distribution can save.
- **Separate compute time from I/O time** — Profile how much of the run is spent reading and writing versus calculating. If I/O dominates, adding workers adds waiting rather than throughput.

## Confirm the work can be split

- **Check that units are genuinely independent** — Each unit must produce its result without needing another unit’s output. Anything sequential needs restructuring first: a running total, a dependency chain.
- **Identify shared state** — Look for a global counter, a shared cache, or a file every worker writes to. Shared mutable state is what turns a parallel run into a race condition.
- **Decide where results are combined** — Know how partial results merge, and whether that combine step is itself expensive enough to become the new bottleneck.

## Prepare the data

- **Store inputs where every worker can reach them** — Data on a single machine’s local disk is not visible to the cluster. Inputs need to be in object storage or workspace-persistent storage.
- **Prefer columnar formats for wide data** — A columnar format lets a worker read only the fields it uses. On a wide table, that alone can cut read volume by an order of magnitude.
- **Choose a partition size deliberately** — Too small and scheduling overhead dominates; too large and the final few tasks leave most workers idle. Aim for units that take seconds, not milliseconds.

## Set expectations

- **Write down the target** — Decide what runtime would make this worth doing. Without a target, "faster" has no stopping point and tuning never ends.
- **Estimate the cost of the run** — Worker count multiplied by expected wall-clock time is your compute spend. Check it against the plan limits on your dashboard before launching.