# When I/O is the bottleneck, not compute

:::warning Illustrative figures
The 7 TB / 192,000-file / ~3-minute result is Eugo's published benchmark. The per-file timings,
per-stage shares, and scaling numbers in this lesson are **illustrative**: they show the shape of
the finding rather than measurements from that specific run, and are being confirmed against the
original profiling data.

The reasoning is the lesson. Measure your own workload for numbers you will act on.
:::


The most consequential diagnostic on the platform, because the two regimes respond to opposite
treatments.

**Compute-bound**: limited by calculation speed. More workers helps.

**I/O-bound**: limited by reading and writing. More workers adds *waiting*.

Our workload was 52% I/O. That determined everything about how it scaled.

## How to tell

Profile one unit and compare the stage shares, as in
[profile before you parallelize](./02-profiling-first.mdx). Read plus write above roughly 40% means I/O will
constrain your scaling.

The empirical check: double the workers and measure.

| Observation | Regime |
| --- | --- |
| Runtime roughly halves | Compute-bound. Keep scaling. |
| Runtime barely moves | I/O-bound. Stop adding workers. |

## Why more workers stops helping

[Object storage](/glossary/object-storage) is a **shared resource**. Every worker reads through the same aggregate bandwidth.

Compute scales with workers because each has its own cores. Read bandwidth does not. Adding a worker
gives it a share of the same pipe. Past the point where that pipe is saturated, each new worker gets a
thinner slice, and the total stays flat while cost keeps rising.

From the [speedup explorer](/interactive):

:::warning Figures pending verification
The speedup figures below are **illustrative** — no capture backs them. They show the *shape* of
how each regime scales, not measurements of Eugo hardware. Do not quote them as Eugo performance
figures; measure your own workload.
:::

| Workers | Compute-bound | I/O-bound |
| --- | --- | --- |
| 4 | 3.8x | 2.7x |
| 8 | 7.2x | 3.7x |
| 16 | 13.4x | 4.4x |
| 32 | 23.6x | 4.7x |
| 64 | 38.2x | 4.7x |

The I/O column is the lesson. Between 32 and 64 workers it gains **nothing**: twice the cost, the same
runtime.

## What actually helps an I/O-bound workload

Reduce bytes moved, or overlap the waiting.

### Read less

**Columnar formats.** Fetch only the fields you use. On a wide table this alone can be an order of
magnitude.

**Predicate pushdown.** Filter at the storage layer rather than reading everything and discarding.

**Appropriate precision.** `float32` instead of `float64` halves the volume, where the problem tolerates
it.

### Overlap the waiting

Within a task, read the next file while processing the current one:

```python
from concurrent.futures import ThreadPoolExecutor

@eugo.hpc.distribute
def process_batch(paths):
    results = []
    with ThreadPoolExecutor(max_workers=4) as pool:
        # Reads are I/O-bound, so threads overlap them despite the GIL
        for raster in pool.map(read_from_object_storage, paths):
            results.append(handle(raster))
    return results
```

Threads work here precisely because the bottleneck is waiting on the network, not holding the GIL. This
raises per-worker throughput without adding workers, the right lever in this regime.

### Right-size the requests

Very small reads pay per-request overhead repeatedly; very large ones delay the start of processing.
[Partition](/glossary/partition) so each read moves a useful amount. See [partitioning](./03-partitioning.mdx).

## What does not help

**More workers.** Covered above.

**GPUs.** A GPU cannot accelerate waiting. Requesting one for an [I/O-bound](/glossary/io-bound) task pays the node premium for
an idle device.

**Optimizing the compute stage.** Our resampling was 17% of runtime. Making it free would have saved 17%,
and I/O would still dominate.

## Where we landed

We ran at **16 workers**, not 64.

Beyond 16, added workers contended for the same read bandwidth and the curve flattened. Sixty-four
workers would have cost four times as much for roughly the same three minutes.

The right worker count is where the curve stops bending, not the maximum your plan permits. The
[cluster sizer](/interactive) estimates that point from your workload's shape.

## The diagnostic, in order

1. Profile one unit; get read/write share of runtime
2. Above ~40% I/O, expect early flattening
3. Double workers, measure, confirm which regime you are in
4. [Compute-bound](/glossary/compute-bound) → keep scaling. I/O-bound → reduce bytes, overlap reads, stop adding workers
5. Record the worker count where gains stop

[What we measured](./05-what-we-measured.mdx) has the full numbers from this run.

---

**Video:** [Reading large datasets without stalling](/videos/reading-large-datasets), the same material with a transcript.

---

Source: https://university.eugo.io/lesson/scaling-a-real-workload/io-bound-vs-cpu-bound