# Requesting GPUs for a task

Automatic offloading needs a GPU to offload *to*. That is a scheduling request:

```python
@eugo.hpc.distribute
def heavy_math(array):
    return expensive_matrix_work(array)

future = heavy_math.options(num_gpus=1)(array)
```

## The request is a constraint

`num_gpus=1` tells the [scheduler](/glossary/scheduler) this task requires a node with a free GPU. Until one exists, the task
waits.

That produces a recognizable symptom: **tasks submitted, none running**. Almost always a resource
request nothing on the [cluster](/glossary/cluster) can satisfy, commonly a GPU request on a cluster provisioned without
GPU nodes. Check GPU availability for your plan on the [dashboard](/glossary/dashboard) before designing around it.

## Fractional GPUs

A single inference call rarely saturates a modern device. Several tasks can share one:

```python
futures = [infer.options(num_gpus=0.25)(x) for x in items]
```

Four tasks per GPU, four-way parallelism, one device.

:::warning Fractional allocation divides scheduling, not memory
`num_gpus=0.25` lets four tasks co-schedule on one device. It does not [partition](/glossary/partition) VRAM. Four tasks each
wanting 20 GB on a 24 GB card will fail with out-of-memory errors, not queue politely.
:::

Size fractions by memory footprint, not by convenience.

## Mixed pipelines

Request GPUs stage by stage:

```python
# Parse: CPU work, no GPU
parsed = [parse.options(num_cpus=2)(f) for f in files]

# Transform: high arithmetic intensity, GPU earns its cost
transformed = [heavy_math.options(num_gpus=1)(p) for p in parsed]

# Summarize: cheap, CPU
summary = [summarize.options(num_cpus=1)(t) for t in transformed]
```

## What over-requesting costs

```python
# Bad: every task demands a GPU, including string parsing
futures = [step.options(num_gpus=1)(x) for x in items]
```

Two separate costs:

**Throughput.** CPU-only tasks queue behind GPU availability. The cluster has idle CPU capacity and
your run is slower than it would be without the request.

**Money.** GPU nodes carry a premium. Paying it for work that never touches the device is waste with no
upside.

## CPUs matter too

`num_cpus` is worth setting when a task is internally multi-threaded, as many NumPy operations are.
Without it a task gets roughly one core's share and its threads contend:

```python
future = numpy_heavy.options(num_cpus=8)(data)
```

Requesting more cores than the task can use is the same mistake as over-requesting GPUs, in a smaller
denomination.

## Combining both

```python
future = train_step.options(num_cpus=4, num_gpus=1)(batch)
```

Typical for training: the GPU does the arithmetic while CPU cores handle data loading and augmentation
feeding it.

## Then verify

A request is not proof of use. [Confirming your work ran on a GPU](./03-verifying-gpu-use.mdx) covers how to
check.

---

Source: https://university.eugo.io/lesson/gpu-acceleration-on-eugo/requesting-gpus