Performance & optimization
Automatic optimizations, profiling, and getting results faster.
Courses
Distributed Python with eugo.hpc
Turn ordinary Python into distributed work with a decorator, then scale it across a cluster.
GPU acceleration on Eugo
How work reaches the GPU, which operations benefit, and how to confirm it actually happened.
How automatic optimizations work
Eugo rewrites and tunes your code as it runs. Here is what it does, and how to help it.
Scaling a real workload
Take a single-machine geospatial pipeline to a cluster, and find out where the time actually goes.
Videos
Coming soonWorking with futures without blocking
The most common performance mistake in distributed code, and the two-line habit that avoids it.
Coming soonAutomatic GPU offloading
Which array operations Eugo moves to the GPU on its own, and how to confirm it happened.
Coming soonParallel pandas in practice
Split a dataframe workload across workers without rewriting your analysis.
Coming soonReading large datasets without stalling
When I/O is your bottleneck, more workers will not help. Here is what does.
Coming soonSizing a cluster for your workload
Estimating worker count from the shape of your work, rather than guessing.
Coming soonSeven terabytes in three minutes
A real geospatial pipeline: 192,000 satellite files, resampled and written, start to finish.
Checklists and guides
Workload readiness checklist
Verify a workload is worth distributing before you spend cluster time on it.
Performance tuning checklist
Work through the usual causes when a distributed run is slower than expected.
Cost optimization checklist
Keep compute spend predictable without giving up throughput: what to check weekly, how to stop paying for idle clusters, and where to right-size.
Quick guide: ML engineer
GPUs, per-call resource requests, and which optimizations the runtime applies on your behalf, plus how to confirm work actually reached the GPU.
Vocabulary
The terms this subject uses, defined.
- Parallel computing : Many calculations at once rather than one after another. Two scales on Eugo: across machines in a cluster, and inside a CPU core through vectorization.
- ARM Neoverse : The ARM server processor architecture Eugo’s compute nodes are built on, chosen for performance per watt on numeric workloads.
- Batching : Grouping several small units of work into one task so that scheduling overhead does not dominate the time spent doing useful work.
- Columnar format : A file format that stores values grouped by column rather than by row, so a reader can fetch only the fields it needs.
- Compute-bound : A workload whose runtime is limited by how fast calculations can be performed. Adding workers to a compute-bound job generally makes it finish sooner.
- Fan-out : Launching many independent tasks at once, then waiting on all of their results together.
- get() : The eugo.hpc call that resolves one or more futures into actual values, blocking until they are ready.
- Blocking : Waiting for a result before continuing. In distributed code, blocking too early is the most common cause of a run that gains no speedup.
- GPU offloading : Moving computationally intensive operations onto a GPU. Eugo applies this automatically when a GPU is available and the data transfer is worth its cost.
- I/O-bound : A workload whose runtime is limited by reading and writing data rather than by calculation. Adding workers to an I/O-bound job adds waiting, not throughput.
- Long tail : The situation where most tasks finish quickly but one or two run much longer, so the whole run waits on them while workers sit idle.
- Automatic optimization : Transformations Eugo applies to your code as it runs: vectorization, GPU offloading, and low-level tuning, with no annotation from you.
- Partition : One independently processable chunk of a dataset. Partition size is the main lever on how well a workload parallelizes.
- Profiling : Measuring where a program actually spends its time, before deciding what to optimize.
- Scheduling overhead : The fixed cost of dispatching a task and returning its result. When it approaches the cost of the task body, adding parallelism stops helping.
- Serialization : Converting Python objects into bytes so they can be sent to another machine. Large task arguments are serialized and copied to every worker that needs them.
- Speedup : How many times faster a distributed run is than the same work on one worker. Ideal speedup equals worker count; real speedup falls short of it.
- Straggler : A single task that runs far longer than its peers and holds up completion of the whole run.
- Transfer cost : The time spent moving data between machines, or between host and GPU memory. Often the reason an operation is not worth accelerating.
- Vectorization : Rewriting operations so a single instruction processes several data elements at once, using the processor’s wide registers.
- Idle cluster : A cluster still running after its work finished. It bills at the full rate, and is the largest avoidable cost for most organizations.