# Feel the problem for yourself

Arguments about why a platform is worth using are cheap. Ten minutes of measuring is not.

You run the same work twice: once the way you already would, once distributed. Then you compare the
two numbers. Everything the rest of the course teaches follows from what you see here.

## Set up

Open a session and a fresh [notebook](/glossary/notebook). Paste this:

```python
import time
import numpy as np

def process(seed):
    """Stand-in for a real per-item workload: some numeric work, a few hundred ms of it."""
    rng = np.random.default_rng(seed)
    a = rng.random((900, 900))
    return float((a @ a.T).trace())

ITEMS = list(range(200))
```

Nothing distributed yet. `process` is deliberately ordinary, the kind of function that shows up in
any analysis, doing real arithmetic over a real array.

## Run it the way you would today

```python
start = time.perf_counter()
serial = [process(i) for i in ITEMS]
print(f"{len(serial)} items in {time.perf_counter() - start:.1f}s")
```

Let it finish. Write the number down.

On a single machine this takes **on the order of a minute or two**: 200 items at a few hundred
milliseconds each. The exact figure depends on your session size, and that is deliberate. It is
*your* baseline, not a number from a benchmark table.

:::note What you are measuring
One core, one item at a time. The 199 items not currently being processed are waiting.
:::

## Now distribute it

Launch a [cluster](/glossary/cluster) from the [EugoHPC Manager](/glossary/eugohpc-manager) panel with 16 workers, which is a good size for this. Wait
for it to report ready.

Then change three lines:

```python
import eugo.hpc

@eugo.hpc.distribute          # 1. the function becomes a task
def process(seed):
    rng = np.random.default_rng(seed)
    a = rng.random((900, 900))
    return float((a @ a.T).trace())

start = time.perf_counter()
futures = [process(i) for i in ITEMS]      # 2. calls return immediately
results = eugo.hpc.get(futures)            # 3. one wait, at the end
print(f"{len(results)} items in {time.perf_counter() - start:.1f}s")
```

Run it. Write the second number down.

## What just happened

**The body of `process` did not change.** Same NumPy, same arithmetic. If you diff the two versions,
the function is byte-identical apart from the decorator above it.

**The call site barely changed.** `process(i)` still looks like calling a function. What changed is
what it *returns*. Instead of a value you get a **future**, a handle to a result that does not exist
yet. That is why the list comprehension finishes almost instantly while the work is still running.

**One wait, not two hundred.** `eugo.hpc.get()` on the whole list blocks once, after every task is
already in flight.

## The number you should not expect

Your second measurement will be **meaningfully faster, and nowhere near 16×**.

That gap matters more than either number on its own. It comes from real costs:

- Dispatching 200 tasks is not free.
- The cluster had to hand each worker its arguments and collect each result.
- The last few tasks finish while most workers are already idle.

A platform that claimed 16× would be lying to you. Knowing the shape of the gap is what lets you
size a cluster instead of guessing. [Parallel fan-out](/lesson/distributed-python-with-eugo-hpc/parallel-fan-out)
is where you learn to close it.

## Try breaking it

Worth two minutes, because it is the mistake that costs people the most:

```python
# Move get() inside the loop
start = time.perf_counter()
results = [eugo.hpc.get(process(i)) for i in ITEMS]
print(f"{len(results)} items in {time.perf_counter() - start:.1f}s")
```

This is **slower than the serial version**. You now pay dispatch overhead on every item while still
running them one at a time. Nothing errors. Nothing warns you.

That is why [futures and `get()`](/lesson/distributed-python-with-eugo-hpc/futures-and-get) gets a
lesson of its own.

## Shut the cluster down

From the same panel you launched it. A cluster bills for as long as it runs, whether or not it is
doing anything. That is the most common avoidable cost on the platform.

## What you now know

- Distributing existing Python is a decorator and one `get()`, not a rewrite.
- A distributed call returns a future, not a value.
- Real [speedup](/glossary/speedup) falls short of worker count, for reasons you can name.
- `get()` in the wrong place makes things worse, silently.

Next: [open your first interactive session](./04-first-session.mdx) properly, and find out which of
your files survive a restart.

---

Source: https://university.eugo.io/lesson/getting-started-with-eugo/why-eugo