# Vectorization and the ARM advantage

**Vectorization** means one instruction processing several data elements at once, using the processor's
wide registers. Sixteen additions in the time of one.

## The idea

A scalar loop adds one pair per instruction:

```
a[0] + b[0]     →  one instruction
a[1] + b[1]     →  one instruction
...
```

A vector instruction loads several elements into a wide register and operates on all of them together:

```
a[0..7] + b[0..7]   →  one instruction
```

Same result, a fraction of the instructions. This is SIMD (single instruction, multiple data), and it
is the reason `np.sum(values ** 2)` outruns the equivalent Python loop by a wide margin.

## Why ARM Neoverse

Eugo's compute nodes are built on ARM Neoverse cores, and Eugo maintains a custom LLVM and Clang
toolchain targeting them.

Two things follow:

**Performance per watt.** Neoverse cores deliver strong throughput on numeric work at lower power than
comparable alternatives, which is why the platform can offer the capacity it does at the price it does.

**A toolchain we control.** Because Eugo builds its own compiler stack rather than relying on generic
builds, vectorization is applied to ordinary numeric Python rather than only to code someone
hand-tuned. Eugo also contributes upstream to the projects this depends on.

## What blocks vectorization

Worth recognizing, because these are the patterns that leave performance on the table.

**Loop-carried dependencies.** Each iteration needing the previous result forces sequential execution:

```python
# Cannot vectorize: every step depends on the last
for i in range(1, n):
    values[i] = values[i - 1] * decay + inputs[i]
```

This is a genuine dependency, not a coding mistake. Some such recurrences have parallel formulations;
many do not.

**Branching per element.** Different elements taking different paths defeats uniform execution:

```python
# Divergent
for i in range(n):
    out[i] = expensive(x[i]) if x[i] > threshold else cheap(x[i])

# Vectorizable: compute both, select
out = np.where(x > threshold, expensive_vec(x), cheap_vec(x))
```

The second does more arithmetic and is usually faster anyway, because it stays in vector form.

**Irregular memory access.** Vector loads want contiguous elements. Random indexing or heavy striding
means the loads cannot be coalesced.

**Calling into opaque Python.** A vectorizer cannot see through an arbitrary Python callback:

```python
# Row-wise Python call: a barrier
df["out"] = df["value"].apply(my_python_function)

# Array operation: optimizable
df["out"] = np.log1p(df["value"]) * 2
```

`apply` with a Python function is the most common performance mistake in pandas code, and no amount of
platform optimization removes it.

## Precision

Narrower types fit more elements per register:

```python
data = data.astype(np.float32)     # twice as many per vector op as float64
```

`float32` is standard for [GPU](/glossary/gpu) work and adequate for most machine learning and much scientific
computing. Confirm your problem tolerates it. Some numerical methods genuinely need double precision, and a
silently wrong answer is worse than a slow right one.

## What to take from this

You do not enable vectorization. You avoid [blocking](/glossary/blocking) it: prefer array operations over element loops, use
`np.where` over per-element branches, keep memory access contiguous, and keep Python callbacks out of
hot paths.

[Writing code the optimizer can help](./03-writing-optimizable-code.mdx) turns this into practice.

---

Source: https://university.eugo.io/lesson/how-automatic-optimizations-work/vectorization