Skip to main content

Notebooks, files, and what persists

The single most useful thing to know about EugoIDE: not everything you write to disk stays there.

The rule

Files in the workspace tree, which is what the file browser shows, persist between sessions. Everything else is session-local and disappears when the session ends.

df.to_parquet("/workspace/results/output.parquet") # persists
df.to_parquet("/tmp/output.parquet") # gone on restart

Three storage locations, three purposes

Workspace storage. Notebooks, scripts, small reference data. Visible to your session. Persists.

Object storage. Large inputs and outputs. Reachable by every compute node in parallel, which makes it the right place for anything workers read.

Session-local (/tmp). Scratch space within one session. Fast, ephemeral, and invisible to the cluster.

A path that works in your notebook may not work in a task

Your session can read workspace storage. A worker on a compute node generally cannot. If a task needs data, that data belongs in object storage.

This is the error that produces a confusing FileNotFoundError inside a task while the same path reads fine in the notebook cell above it.

Organizing a workspace

A layout that holds up:

/workspace/
├── notebooks/ exploratory work
├── src/ reusable modules, imported by notebooks
├── data/ small reference files only
└── results/ local outputs worth keeping

Large inputs and outputs go to object storage, not into data/.

Notebooks are not the only option

Long-lived logic belongs in a .py module under src/, imported by the notebook. Notebooks are for exploration and narrative; modules are for code you will reuse or test.

import sys
sys.path.insert(0, "/workspace/src")
from pipeline import transform

Version control

There is no automatic history. If work matters, commit it. A terminal is available, and git works as usual against your workspace tree.

Before you close a session

Check for anything written outside the workspace tree. Once the session ends, it is not recoverable.