import { BlockingComparison } from '@site/src/components/diagrams/BlockingComparison';

# Futures, and when to call get()

A **future** is a handle to a result that does not exist yet. This lesson is about one decision: where
you call `get()`.

Getting it wrong costs more parallelism than any other mistake in this course.

## What a future is

```python
future = transform(record)
print(type(future))                 # a handle, not your result
print(eugo.hpc.get(future))         # the value
```

Calling a distributed function returns immediately. The work happens on a [worker](/glossary/worker) while your code keeps
going. `get()` blocks until the result is ready.

## The mistake

```python
# WRONG: runs one task at a time
results = []
for record in records:
    results.append(eugo.hpc.get(transform(record)))
```

Trace it: dispatch one [task](/glossary/task), **wait** for it, dispatch the next, wait again. Only one task is ever in
flight. Your [cluster](/glossary/cluster) of 32 workers has 31 idle at all times, and you are paying for all of them.

## The fix

```python
# RIGHT: every task in flight before waiting on any
futures = [transform(record) for record in records]
results = eugo.hpc.get(futures)
```

Build all the futures, then resolve once.

<BlockingComparison />

The difference is moving one call out of a loop. On a 32-worker cluster it is frequently the difference
between no [speedup](/glossary/speedup) and roughly 25x.

:::tip The rule
Build every future first. Call `get()` once, on the list.
:::

## Why this is so easy to get wrong

The broken version looks reasonable. It reads like ordinary Python, produces correct results, and
nothing errors. It is only slow. In a distributed system, slowness is easy to blame on the cluster,
the data, or the network rather than on your own loop.

When a run disappoints, check this first. It accounts for more missing speedup than everything else
combined.

## Passing futures to tasks

You can pass a future as an argument. Eugo resolves it before the body runs, which lets you express a
dependency:

```python
@eugo.hpc.distribute
def stage_one(x):
    return x * 2

@eugo.hpc.distribute
def stage_two(value):
    return value + 1

chained = stage_two(stage_one(10))     # no get() in between
print(eugo.hpc.get(chained))           # 21
```

The intermediate result never returns to your session. It moves directly between workers, saving a
round trip.

## Accidental serialization

The flip side: a genuine dependency chain cannot parallelize.

```python
# Each task needs the previous result. Serial by construction.
running = seed
for x in items:
    running = accumulate(running, x)
result = eugo.hpc.get(running)
```

No cluster size fixes this. The work has to be restructured, often as a tree reduction that combines
pairs in parallel rather than folding left.

## Partial results

To handle results as they arrive rather than waiting for all:

```python
futures = [transform(r) for r in records]

while futures:
    done, futures = eugo.hpc.wait(futures, num_returns=1)
    for f in done:
        handle(eugo.hpc.get(f))
```

Useful for streaming output or showing progress. For most work, one `get()` on the full list is
simpler and just as fast.

## What to remember

- A future is a promise, not a value.
- `get()` inside a loop serializes your run.
- Build all futures, then `get()` once.
- Chained futures avoid a round trip through your session.

---

**Video:** [Working with futures without blocking](/videos/futures-without-blocking). The same material, with a transcript.

---

Source: https://university.eugo.io/lesson/distributed-python-with-eugo-hpc/futures-and-get