Eugo for research scientists
Run simulations and scientific workloads at scale
Courses for this role
Ordered as they appear in the catalog; any of them can be taken on its own.
Getting started with Eugo
What Eugo is, what problem it solves, and how to run your first workload on a cluster.
Beginner31m6 lessons
EugoIDE essentials
Work fluently in the browser-based notebook environment where your Python actually runs.
Beginner30m5 lessons
GPU acceleration on Eugo
How work reaches the GPU, which operations benefit, and how to confirm it actually happened.
Intermediate26m4 lessons
How automatic optimizations work
Eugo rewrites and tunes your code as it runs. Here is what it does, and how to help it.
Intermediate29m4 lessons
Scaling a real workload
Take a single-machine geospatial pipeline to a cluster, and find out where the time actually goes.
Advanced38m5 lessons
What you will actually cover
Every lesson in those courses, so you can see the ground before committing to it.
Vocabulary you will meet
The terms these courses use, defined.
- High-performance computing (HPC): Running work across many machines at once so it finishes far sooner. The technique behind weather models, genomics, and large-scale simulation.
- HPC as a service (HPCaaS): High-performance computing consumed on demand rather than owned: request compute for one run, stop paying when it ends. No hardware, no scheduler to run.
- Distributed computing: Splitting one job across several machines that work on it simultaneously. The core technique of HPC, and what the distribute decorator gives you in Python.
- Job scheduler: The queueing system on a traditional HPC cluster that decides when a submitted job runs. Eugo has no queue: you launch a cluster and dispatch work to it.
- Parallel computing: Many calculations at once rather than one after another. Two scales on Eugo: across machines in a cluster, and inside a CPU core through vectorization.
- Actor: A distributed object that holds state across many calls. Created by applying the distribute decorator to a class rather than a function.
- ARM Neoverse: The ARM server processor architecture Eugo’s compute nodes are built on, chosen for performance per watt on numeric workloads.
- Batching: Grouping several small units of work into one task so that scheduling overhead does not dominate the time spent doing useful work.
- Cluster: A set of machines, one head node and one or more compute nodes, launched together to run distributed work.
- Columnar format: A file format that stores values grouped by column rather than by row, so a reader can fetch only the fields it needs.
- Compute node: A machine in a cluster that executes tasks. Compute nodes do the work; the head node coordinates it.
- Compute-bound: A workload whose runtime is limited by how fast calculations can be performed. Adding workers to a compute-bound job generally makes it finish sooner.
- distribute decorator: The eugo.hpc decorator that turns a function into a distributed task, or a class into an actor. A decorated call returns a future instead of a value.
- EugoHPC Manager: The panel inside EugoIDE used to launch, monitor, and shut down clusters.
- EugoIDE: The browser-based notebook environment where you write and run Python on Eugo. Built on JupyterLab, connected to compute in your workspace.
- Fan-out: Launching many independent tasks at once, then waiting on all of their results together.
- Future: A handle to a result that does not exist yet. Calling a distributed function returns a future immediately, while the work happens elsewhere.
- get(): The eugo.hpc call that resolves one or more futures into actual values, blocking until they are ready.
- Blocking: Waiting for a result before continuing. In distributed code, blocking too early is the most common cause of a run that gains no speedup.
- GPU offloading: Moving computationally intensive operations onto a GPU. Eugo applies this automatically when a GPU is available and the data transfer is worth its cost.
- GPU: A graphics processing unit. Hardware with many parallel cores, well suited to large array and matrix operations.
- Head node: The machine in a cluster that coordinates work: it schedules tasks onto compute nodes and tracks their results.
- Interactive session: A running EugoIDE instance attached to your workspace. It is where you write code; a cluster is what runs it at scale.
- I/O-bound: A workload whose runtime is limited by reading and writing data rather than by calculation. Adding workers to an I/O-bound job adds waiting, not throughput.
- Long tail: The situation where most tasks finish quickly but one or two run much longer, so the whole run waits on them while workers sit idle.
- Object storage: Storage reachable by every node in a cluster. Inputs must live here, or in workspace storage, rather than on one machine’s local disk.
- options(): The method that sets per-call resource requirements on a distributed function, such as num_cpus or num_gpus.
- Automatic optimization: Transformations Eugo applies to your code as it runs: vectorization, GPU offloading, and low-level tuning, with no annotation from you.
- Partition: One independently processable chunk of a dataset. Partition size is the main lever on how well a workload parallelizes.
- Persistent storage: Workspace storage that survives between interactive sessions. Files saved elsewhere in a session are lost when it ends.
- Profiling: Measuring where a program actually spends its time, before deciding what to optimize.
- Scheduler: The component on the head node that decides which compute node runs each task, honoring the resources that task requested.
- Scheduling overhead: The fixed cost of dispatching a task and returning its result. When it approaches the cost of the task body, adding parallelism stops helping.
- Serialization: Converting Python objects into bytes so they can be sent to another machine. Large task arguments are serialized and copied to every worker that needs them.
- Speedup: How many times faster a distributed run is than the same work on one worker. Ideal speedup equals worker count; real speedup falls short of it.
- Straggler: A single task that runs far longer than its peers and holds up completion of the whole run.
- Task: One unit of distributed work: a single call to a function decorated with the distribute decorator. Tasks are stateless, so each call starts fresh.
- Transfer cost: The time spent moving data between machines, or between host and GPU memory. Often the reason an operation is not worth accelerating.
- Vectorization: Rewriting operations so a single instruction processes several data elements at once, using the processor’s wide registers.
- Worker: A process on a compute node that executes tasks. Worker count, not node count, is what determines how many tasks run at once.
- Workspace: An isolated compute environment with its own persistent storage, preinstalled libraries, and resource allocation. Where your sessions and clusters run.
- Notebook: A document interleaving code, output, and prose. The primary interface for writing Python in EugoIDE.
- Shared state: Data that more than one task reads or writes. Shared mutable state is what turns a parallel run into a race condition.
- Idle cluster: A cluster still running after its work finished. It bills at the full rate, and is the largest avoidable cost for most organizations.
- Independent work: Units of work that can each produce their result without needing another unit’s output. A precondition for distributing anything.