Skip to main content

Getting extra packages onto the workers

Preinstalled libraries showed %pip install adding a package to your session. That package is not on the cluster. This lesson covers why, what the workers already have, and how to send them anything extra.

Two environments, installed separately​

Your EugoIDE session and the cluster's workers are two separate Python environments, and installing into one does not install into the other:

EnvironmentWhat it isWhat it is for
Your sessionThe Python your notebooks and terminal run onDevelopment, exploration, iterating
The cluster's workersThe Python on every node of your clusterRunning the work at scale

A %pip install in a notebook, or a pip install in a terminal, changes only your session. It does not reach the workers, not even when you install into a folder in the workspace tree: the workers do not see your session's files. To change what the workers see, you pass a runtime environment (runtime_env) on the function that needs it, or when you connect to the cluster, and Ray sets it up on the workers.

That is deliberate. Your session can carry half-finished experiments and extra debugging tools without forcing every node of the cluster to match, and you ship to the cluster on purpose.

It is also the cause of a familiar surprise: an import error inside a task while the same import works in the notebook cell above it.

An install in your session also lasts only as long as the session. Restarting the kernel keeps it; a new session starts from the image again, because Python's packages folder is not in the workspace tree, where the files that are kept live. pip install --user is no different: its folder is outside the workspace tree too. Keep the install line in your notebook's first cell, or ask Eugo to add the package to the image.

What the workers already have​

Every node of an Eugo cluster runs the same image as your session, so a task sees Eugo's preinstalled packages with no runtime_env at all.

Calling a distributed function

Call a function decorated with @eugo.hpc.distribute with .remote(), and read the result with eugo.hpc.get; calling it directly raises a TypeError. @eugo.hpc.distribute is Ray's @ray.remote under Eugo's name, and eugo.hpc.get is ray.get.

How to run the examples on this page

The examples call eugo.hpc.init() with no address, so Ray runs inside your session rather than on your cluster, which keeps their output reproducible. Ray inside your session also sees what you installed in the session, so a run like this does not show what your cluster's workers can import. Run them with no cluster connected: if you clicked Connect, click Disconnect first. Restart the kernel before each example, or run eugo.hpc.shutdown() first: a second eugo.hpc.init in the same kernel raises a RuntimeError. Connecting your own cluster comes at the end of the page.

import eugo.hpc, os
eugo.hpc.init()

@eugo.hpc.distribute
def what_worker_sees():
import numpy, ray as r, sys, os
return {"py": sys.version.split()[0], "numpy": numpy.__version__, "ray": r.__version__,
"numpy_file": numpy.__file__, "pid": os.getpid()}

print("DRIVER pid:", os.getpid(), "numpy:", __import__("numpy").__version__)
print("WORKER sees:", eugo.hpc.get(what_worker_sees.remote()))
DRIVER pid: 188 numpy: 2.6.0.dev0+git20260704.c6dcb75
WORKER sees: {'py': '3.14.6', 'numpy': '2.6.0.dev0+git20260704.c6dcb75',
'ray': '3.0.0.dev0', 'numpy_file': '/usr/local/lib/python3.14/site-packages/numpy/__init__.py', 'pid': 752}

The worker (pid 752) is a separate process from the driver that called it (pid 188), and it finds numpy in Eugo's own build, the same one the driver has.

This run, and the two below, used Ray inside a single Eugo container, so the output is reproducible. The runtime_env behaviour is the same on your cluster. The version numbers are the ones that image shipped; yours may differ.

You do not need to list preinstalled packages in a runtime_env. Only what is missing.

Sending extra packages: a pip list​

For a package the image does not have, give Ray a pip list. Ray builds a fresh virtual environment for the job from that list, on top of Eugo's packages (it is created with the system site-packages), so the workers keep everything the image provides and gain your extras:

import eugo.hpc
eugo.hpc.init(runtime_env={"pip": ["cowsay==6.1"]})

@eugo.hpc.distribute
def t():
import cowsay, sys
return {"cowsay": cowsay.__version__, "py": sys.version.split()[0]}

print(eugo.hpc.get(t.remote()))
{'cowsay': '6.1', 'py': '3.14.6'}

The package installed into the job's environment, and the worker imported it.

The package has to be reachable​

The install still has to fetch what you ask for. Either the cluster reaches PyPI (or your own package index), or you point it at a folder of wheel files:

import eugo.hpc
eugo.hpc.init(runtime_env={"pip": {
"packages": ["cowsay==6.1"],
"pip_install_options": ["--no-index", "--find-links", "/path/to/wheelhouse"],
}})

The folder must be at that path on every node of the cluster, for example on shared storage; a folder that exists only in your session is not enough.

Choosing what to send​

  • A pure-Python package on your index: a pip list, as above.
  • A compiled package you want tuned for the hardware (NumPy, PyArrow, and the like): ask Eugo to add it to the image, where it is built for your CPU, rather than pulling a generic wheel from PyPI into the job. Most of the common stack is in the image already.

The same, installed by uv​

Ray can install the same list with uv, Astral's package installer, which is much faster for long lists. It works exactly like pip above: a fresh virtual environment per job, over Eugo's own interpreter and packages. The keys mirror pip's, with uv_pip_install_options in place of pip_install_options. Offline, from a folder of wheels:

import eugo.hpc
eugo.hpc.init(runtime_env={"uv": {
"packages": ["cowsay==6.1"],
"uv_pip_install_options": ["--offline", "--no-index", "--find-links", "/path/to/wheelhouse"],
}})

@eugo.hpc.distribute
def t():
import cowsay, sys
return {"cowsay": cowsay.__version__, "py": sys.version.split()[0], "uv_venv": "/runtime_resources/uv/" in sys.prefix}

print(eugo.hpc.get(t.remote()))
{'cowsay': '6.1', 'py': '3.14.7', 'uv_venv': True}

The image ships uv, and Ray uses that copy for the job. The same rule as pip applies: the packages you list must be reachable, and a wheel folder must be at that path on every node.

conda environments are not supported, because conda is not in the image. Use pip or uv.

Passing a runtime_env to your own cluster​

The examples above ran against Ray inside your session. On the platform, the Connect button on a cluster in the EugoHPC Manager attaches your notebook to that cluster. The button gives a plain connection: today it does not take a runtime_env. After Connect, an eugo.hpc.init(runtime_env=...) with no address raises no error, and it does not change what your cluster's workers get.

Per function, on the Connect connection​

The simplest way is to keep the Connect connection and give the function that needs the package its own runtime_env, on its decorator:

import eugo.hpc

@eugo.hpc.distribute(runtime_env={"pip": ["cowsay==6.1"]})
def t():
import cowsay
return cowsay.__version__

print(eugo.hpc.get(t.remote()))

For a single call, pass it on .options instead: t.options(runtime_env={...}).remote().

For the whole connection, by hand​

When every function needs the same runtime_env, connect by hand, in the same notebook:

  1. Click Connect on the cluster. It inserts and runs a cell, eugo.hpc.init(address="ray://..."); copy the address from that cell.
  2. Click Disconnect. A second connection while the first is still open raises an error.
  3. Run the hand-written connect, with the address you copied and your runtime_env:
import eugo.hpc

eugo.hpc.init(
address="<the address from the cell Connect inserted>",
runtime_env={
"pip": ["cowsay==6.1"],
},
)

Things to know before you rely on it:

  • The address is the one in the cell Connect inserts. Clusters are temporary, so read it each session and never commit it to your code.
  • The cluster's token is required on every connection, a hand-written one included. Connect sets it in your notebook's kernel and it stays set after Disconnect, so connect by hand in the same notebook without restarting the kernel. In a kernel where Connect never ran, the connection is refused.
  • The keys are Ray's, and a misspelled one fails silently. env_var instead of env_vars is accepted without an error and does nothing. Check the spelling; nothing else will.
  • It applies to the whole connection. To give one function its own environment, use the decorator form above.
  • A hand-written connection does not show on the cluster card. The EugoHPC Manager tracks the connections it makes itself, so keep your own note of which cluster a session is attached to.

What you now know​

  • Your session and the cluster's workers are separate Python environments; %pip install reaches only your session.
  • The workers already have every preinstalled package, so a runtime_env lists only what is missing.
  • A pip or uv list in runtime_env installs extras on the workers, as long as the packages are reachable from the cluster.
  • On your cluster, pass the runtime_env on the function, or connect by hand with it after Connect and Disconnect.
  • A compiled package you want tuned belongs in the image, not in a per-job install.

The full reference, with private indexes, wheel folders and the rest of runtime_env, is Installing Python packages in the Eugo documentation.