Skip to main content

Preinstalled libraries and adding your own

Workspaces come with the scientific Python stack already installed. Check what you have before installing anything:

import numpy, pandas, scipy, sklearn, torch
for m in (numpy, pandas, scipy, sklearn, torch):
print(f"{m.__name__:12s} {m.__version__}")

What is included​

NumPy, pandas, SciPy, scikit-learn, and PyTorch, plus common geospatial and imaging libraries. Between them these cover most data analysis, scientific computing, and model training work.

Python is Eugo's own interpreter, the only one in the image. Its version moves with each image, so check yours rather than assuming: import sys; print(sys.version).

What is not supported​

warning

Eugo does not support TensorFlow, spaCy, or JAX.

This is the constraint most likely to derail a migration, so establish it early. If a pipeline depends on one of them, that dependency needs replacing before the rest of the move is worth planning. PyTorch is the supported path for deep learning work.

Adding a package​

%pip install some-package

Your session has one Python, Eugo's own, under /usr/local: python3, pip3, python and pip all resolve to it. So %pip in a notebook and pip in a terminal install into the same place, the system site-packages that holds the preinstalled libraries.

pip knows about those libraries. A dependency Eugo already provides is reported as already satisfied and left alone, and only the missing pieces are downloaded.

A conflicting version replaces Eugo's build

If a requirement excludes the version Eugo ships, for example a package that needs numpy<2.5, pip uninstalls Eugo's build and installs a generic wheel in its place, for the rest of your session.

Check first with a dry run in a terminal. It changes nothing:

$ pip3 install --dry-run "numpy<2.5"
Collecting numpy<2.5
Downloading numpy-2.4.6-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl.metadata (6.6 kB)
Would install numpy-2.4.6

To keep an install like that contained, put it in a virtual environment instead; Installing Python packages in the Eugo documentation shows how.

How long an install lasts​

An install changes your running session only. It does not change the Eugo image: a fresh session starts from the image again, with the preinstalled libraries and nothing you added. Restarting the kernel keeps an install; a new session does not, because Python's packages folder is not in the workspace tree. pip install --user is no different: its folder is outside the workspace tree too. Keep the install line in your notebook, or a requirements file in the workspace tree (see below), and run it at the start of each session.

A fresh session also restores any preinstalled library an install replaced. Do not count on that as a clean-up step, though; contain a conflicting install in a virtual environment from the start.

Workers need it too​

A package your task body imports must be available on the compute nodes, not only in your session. %pip install does not reach them: the cluster's workers are a separate Python environment. Without the package there, a task raises an ImportError while the same import succeeds in the notebook.

The workers already have every preinstalled library. For anything extra, give the function that imports it a runtime_env on its decorator, and Ray installs it on the workers that run that function. This works on the connection made by the Connect button on a cluster in the EugoHPC Manager:

import eugo.hpc

@eugo.hpc.distribute(runtime_env={"pip": ["cowsay==6.1"]})
def t():
import cowsay
return cowsay.__version__

print(eugo.hpc.get(t.remote()))

A runtime_env passed to eugo.hpc.init() with no address does not reach your cluster. Getting extra packages onto the workers covers what the workers already have, the uv variant, and one runtime_env for a whole connection.

Pin what matters​

For reproducible work, record versions:

%pip freeze > /workspace/requirements-lock.txt

Keep that file in version control alongside the code. "It worked in August" is not a recoverable state without it.