When I/O is the bottleneck, not compute
The 7 TB / 192,000-file / ~3-minute result is Eugo's published benchmark. The per-file timings, per-stage shares, and scaling numbers in this lesson are illustrative: they show the shape of the finding rather than measurements from that specific run, and are being confirmed against the original profiling data.
The reasoning is the lesson. Measure your own workload for numbers you will act on.
The most consequential diagnostic on the platform, because the two regimes respond to opposite treatments.
Compute-bound: limited by calculation speed. More workers helps.
I/O-bound: limited by reading and writing. More workers adds waiting.
Our workload was 52% I/O. That determined everything about how it scaled.
How to tell
Profile one unit and compare the stage shares, as in profile before you parallelize. Read plus write above roughly 40% means I/O will constrain your scaling.
The empirical check: double the workers and measure.
| Observation | Regime |
|---|---|
| Runtime roughly halves | Compute-bound. Keep scaling. |
| Runtime barely moves | I/O-bound. Stop adding workers. |
Why more workers stops helping
Object storage is a shared resource. Every worker reads through the same aggregate bandwidth.
Compute scales with workers because each has its own cores. Read bandwidth does not. Adding a worker gives it a share of the same pipe. Past the point where that pipe is saturated, each new worker gets a thinner slice, and the total stays flat while cost keeps rising.
From the speedup explorer:
The speedup figures below are illustrative — no capture backs them. They show the shape of how each regime scales, not measurements of Eugo hardware. Do not quote them as Eugo performance figures; measure your own workload.
| Workers | Compute-bound | I/O-bound |
|---|---|---|
| 4 | 3.8x | 2.7x |
| 8 | 7.2x | 3.7x |
| 16 | 13.4x | 4.4x |
| 32 | 23.6x | 4.7x |
| 64 | 38.2x | 4.7x |
The I/O column is the lesson. Between 32 and 64 workers it gains nothing: twice the cost, the same runtime.
What actually helps an I/O-bound workload
Reduce bytes moved, or overlap the waiting.
Read less
Columnar formats. Fetch only the fields you use. On a wide table this alone can be an order of magnitude.
Predicate pushdown. Filter at the storage layer rather than reading everything and discarding.
Appropriate precision. float32 instead of float64 halves the volume, where the problem tolerates
it.
Overlap the waiting
Within a task, read the next file while processing the current one:
from concurrent.futures import ThreadPoolExecutor
@eugo.hpc.distribute
def process_batch(paths):
results = []
with ThreadPoolExecutor(max_workers=4) as pool:
# Reads are I/O-bound, so threads overlap them despite the GIL
for raster in pool.map(read_from_object_storage, paths):
results.append(handle(raster))
return results
Threads work here precisely because the bottleneck is waiting on the network, not holding the GIL. This raises per-worker throughput without adding workers, the right lever in this regime.
Right-size the requests
Very small reads pay per-request overhead repeatedly; very large ones delay the start of processing. Partition so each read moves a useful amount. See partitioning.
What does not help
More workers. Covered above.
GPUs. A GPU cannot accelerate waiting. Requesting one for an I/O-bound task pays the node premium for an idle device.
Optimizing the compute stage. Our resampling was 17% of runtime. Making it free would have saved 17%, and I/O would still dominate.
Where we landed
We ran at 16 workers, not 64.
Beyond 16, added workers contended for the same read bandwidth and the curve flattened. Sixty-four workers would have cost four times as much for roughly the same three minutes.
The right worker count is where the curve stops bending, not the maximum your plan permits. The cluster sizer estimates that point from your workload's shape.
The diagnostic, in order
- Profile one unit; get read/write share of runtime
- Above ~40% I/O, expect early flattening
- Double workers, measure, confirm which regime you are in
- Compute-bound → keep scaling. I/O-bound → reduce bytes, overlap reads, stop adding workers
- Record the worker count where gains stop
What we measured has the full numbers from this run.
Video: Reading large datasets without stalling, the same material with a transcript.