Skip to main content

Object storage

Storage reachable by every node in a cluster. Inputs must live here, or in workspace storage, rather than on one machine’s local disk.

The one storage location every worker can reach in parallel, which makes it the right home for anything a distributed task reads. Workspace storage is visible to your session but generally not to a worker on a compute node.

Data & I/OWorkspaces

Related terms

  • Workspace: An isolated compute environment with its own persistent storage, preinstalled libraries, and resource allocation. Where your sessions and clusters run.
  • I/O-bound: A workload whose runtime is limited by reading and writing data rather than by calculation. Adding workers to an I/O-bound job adds waiting, not throughput.
  • Persistent storage: Workspace storage that survives between interactive sessions. Files saved elsewhere in a session are lost when it ends.