Skip to main content

Vectorization and the ARM advantage

Vectorization means one instruction processing several data elements at once, using the processor's wide registers. Sixteen additions in the time of one.

The idea​

A scalar loop adds one pair per instruction:

a[0] + b[0]     →  one instruction
a[1] + b[1]     →  one instruction
...

A vector instruction loads several elements into a wide register and operates on all of them together:

a[0..7] + b[0..7]   →  one instruction

Same result, a fraction of the instructions. This is SIMD (single instruction, multiple data), and it is the reason np.sum(values ** 2) outruns the equivalent Python loop by a wide margin.

Why ARM Neoverse​

Eugo's compute nodes are built on ARM Neoverse cores, and Eugo maintains a custom LLVM and Clang toolchain targeting them.

Two things follow:

Performance per watt. Neoverse cores deliver strong throughput on numeric work at lower power than comparable alternatives, which is why the platform can offer the capacity it does at the price it does.

A toolchain we control. Because Eugo builds its own compiler stack rather than relying on generic builds, vectorization is applied to ordinary numeric Python rather than only to code someone hand-tuned. Eugo also contributes upstream to the projects this depends on.

What blocks vectorization​

Worth recognizing, because these are the patterns that leave performance on the table.

Loop-carried dependencies. Each iteration needing the previous result forces sequential execution:

# Cannot vectorize: every step depends on the last
for i in range(1, n):
    values[i] = values[i - 1] * decay + inputs[i]

This is a genuine dependency, not a coding mistake. Some such recurrences have parallel formulations; many do not.

Branching per element. Different elements taking different paths defeats uniform execution:

# Divergent
for i in range(n):
    out[i] = expensive(x[i]) if x[i] > threshold else cheap(x[i])

# Vectorizable: compute both, select
out = np.where(x > threshold, expensive_vec(x), cheap_vec(x))

The second does more arithmetic and is usually faster anyway, because it stays in vector form.

Irregular memory access. Vector loads want contiguous elements. Random indexing or heavy striding means the loads cannot be coalesced.

Calling into opaque Python. A vectorizer cannot see through an arbitrary Python callback:

# Row-wise Python call: a barrier
df["out"] = df["value"].apply(my_python_function)

# Array operation: optimizable
df["out"] = np.log1p(df["value"]) * 2

apply with a Python function is the most common performance mistake in pandas code, and no amount of platform optimization removes it.

Precision​

Narrower types fit more elements per register:

data = data.astype(np.float32)     # twice as many per vector op as float64

float32 is standard for GPU work and adequate for most machine learning and much scientific computing. Confirm your problem tolerates it. Some numerical methods genuinely need double precision, and a silently wrong answer is worse than a slow right one.

What to take from this​

You do not enable vectorization. You avoid blocking it: prefer array operations over element loops, use np.where over per-element branches, keep memory access contiguous, and keep Python callbacks out of hot paths.

Writing code the optimizer can help turns this into practice.