Skip to main content
Quick guideML engineerGPU & accelerationAI/MLPerformance & optimization

Quick guide: ML engineer

GPUs, per-call resource requests, and which optimizations the runtime applies on your behalf, plus how to confirm work actually reached the GPU.

0 of 4 checked(not saved, download a copy to keep)

Download markdown

1What you need to know first

PyTorch, not TensorFlow
Eugo does not support TensorFlow, spaCy, or JAX. Plan around PyTorch.
Ask for GPUs per call
Resource requirements go on the call site via .options(num_gpus=...), not on the function definition. Mixed pipelines should request GPUs only where needed.
Offloading is automatic but conditional
Eugo moves heavy array work to GPUs when the transfer is worth paying for. Small arrays stay on CPU by design. Verify where work ran rather than assuming.

2For long-lived state

Use an actor for an expensive model
If loading the model costs more than the inference, decorate a class instead of a function. The actor holds the loaded model across many method calls.

More like this

All resources
Checklist

Performance tuning checklist

Work through the usual causes when a distributed run is slower than expected.

9 items
Checklist

Workload readiness checklist

Verify a workload is worth distributing before you spend cluster time on it.

11 items
Checklist

Cost optimization checklist

Keep compute spend predictable without giving up throughput: what to check weekly, how to stop paying for idle clusters, and where to right-size.

7 items