Skip to main content

The optimization pass, end to end

You write standard Python. Between submitting a task and it running, the runtime analyzes and transforms the work. This lesson covers what actually happens, because knowing it is what lets you write code the optimizer can help rather than code it must work around.

The stages​

1. Analysis. The runtime inspects the operations in your task body: which arrays, which operations, what shapes and dtypes.

2. Vectorization. Numeric loops are rewritten to use the processor's wide registers, so one instruction processes several elements. Eugo maintains a custom LLVM and Clang toolchain targeting ARM Neoverse for this. Covered in the next lesson.

3. Offload decision. Operations with high arithmetic intensity over large arrays are candidates for the GPU, if one is allocated and the transfer pays for itself. See what gets offloaded.

4. Scheduling. The task is placed on a node satisfying its resource request.

5. Execution. The transformed work runs.

What you are not asked to do​

No decorators beyond @eugo.hpc.distribute. No type annotations for the optimizer. No compilation step. No kernel-tuning parameters.

This is a deliberate trade: less control in exchange for the optimization applying by default rather than when someone remembers to ask.

What it does not do​

Being clear about this matters more than the capabilities list, because misplaced expectations cost more than absent features.

It does not fix algorithmic complexity. An O(n²) approach stays O(n²), vectorized. Choosing a better algorithm is still your job and still the largest available win.

It does not parallelize across machines for you. Distribution is explicit. That is what @eugo.hpc.distribute and fan-out are for. The optimizer works within a task.

It does not accelerate I/O. Waiting on object storage is not computation. See when I/O is the bottleneck.

It does not rescue Python-level loops over scalars. A for loop doing arithmetic one number at a time in interpreted Python has little for a vectorizer to work with. Array operations do.

The practical consequence​

The runtime optimizes array operations. Code expressed as bulk operations over arrays gives it material; code expressed as element-by-element Python does not.

# Little to optimize: interpreted scalar loop
total = 0.0
for i in range(len(values)):
    total += values[i] ** 2

# Optimizable: one array operation
total = np.sum(values ** 2)

Both are correct. The second is what the optimizer can transform, and it reads better too. The fast path and the clear path are the same one here.

Writing code the optimizer can help covers this properly.

Verify, do not assume​

Automatic optimization is not a reason to skip measurement. The runtime makes decisions based on shapes and sizes it observes, and its decision for your data may not be the one you assumed.

Measuring the gain honestly covers how to check without fooling yourself.