# Confirming your work ran on a GPU

Requesting a GPU guarantees the *hardware*. It does not guarantee your work used it, or that using it
helped. Three separate things, each worth checking.

## 1. Was hardware allocated?

```python
@eugo.hpc.distribute
def gpu_check():
    import torch
    return {
        "available": torch.cuda.is_available(),
        "count": torch.cuda.device_count(),
        "name": torch.cuda.get_device_name(0) if torch.cuda.is_available() else None,
    }

print(eugo.hpc.get(gpu_check.options(num_gpus=1)()))
```

`available: False` with `num_gpus=1` requested means the [task](/glossary/task) did not land on a GPU node. Check plan
availability on the [dashboard](/glossary/dashboard).

## 2. Did your work use it?

For PyTorch, tensors carry their device:

```python
@eugo.hpc.distribute
def where_did_it_run(array):
    import torch
    t = torch.from_numpy(array)
    result = expensive_matrix_work(t)
    return {"input": str(t.device), "output": str(result.device)}
```

`cpu` for both, on a node where CUDA was available, means the work stayed on the host. Usually the
array was too small for offloading to pay, which is the runtime making a correct decision.

## 3. Was it actually faster?

The check people skip, and the only one that matters commercially.

```python
import time

@eugo.hpc.distribute
def timed(array):
    start = time.perf_counter()
    result = expensive_matrix_work(array)
    return {"seconds": time.perf_counter() - start, "shape": result.shape}

cpu = eugo.hpc.get(timed(array))
gpu = eugo.hpc.get(timed.options(num_gpus=1)(array))

print(f"CPU {cpu['seconds']:.3f}s   GPU {gpu['seconds']:.3f}s   "
      f"speedup {cpu['seconds'] / gpu['seconds']:.1f}x")
```

Run each a few times and take the median. First-call timings include device initialization and are
misleading.

:::warning GPU calls are asynchronous
CUDA operations queue rather than block. Timing around them without synchronizing measures how fast you
*submitted* work, not how long it took.

```python
result = expensive_matrix_work(t)
torch.cuda.synchronize()          # wait for completion before stopping the clock
elapsed = time.perf_counter() - start
```

Missing `synchronize()` produces impossibly good numbers. If a [speedup](/glossary/speedup) looks too good, this is usually
why.
:::

## Expect a slowdown sometimes

A speedup below 1x is a legitimate result, not a bug. From the
[illustrative chart data](/interactive) — figures pending verification, so read the ratio as a
shape rather than as a measurement:

- Matmul at 256²: about 2.7x **slower** on the GPU than on the same machine's CPU
- Elementwise over 10K elements: about 17x **slower**, same comparison

[Transfer cost](/glossary/transfer-cost) exceeded compute saved. Finding this is the check working: you now know to leave that
stage on CPU and stop paying the GPU premium for it.

## What to record

For each stage you consider accelerating:

| | Value |
| --- | --- |
| CPU time (median of 3) | |
| GPU time (median of 3, synchronized) | |
| Speedup | |
| Verdict: keep GPU? | |

Without those numbers, "we use GPUs" is an architecture claim rather than a performance one.

## Next

[Transfer costs and when GPUs lose](./04-transfer-costs.mdx) goes into why the boundary sits where it does,
and how to shift it.

---

Source: https://university.eugo.io/lesson/gpu-acceleration-on-eugo/verifying-gpu-use