Skip to main content
Not yet recorded

This walkthrough is planned but not published yet. The transcript below covers the same material if you would rather read it now.

Sizing a cluster for your workload

IntermediateComing soonPerformance & optimizationBilling & plans

Estimating worker count from the shape of your work, rather than guessing.

What this video covers

Start from the number of independent units of work and how long one takes. If you have a thousand partitions at two seconds each, that is roughly thirty-three minutes of serial work; sixteen workers should bring it near two minutes, minus overhead. Scaling stops paying when per-task overhead approaches per-task work, or when a shared resource becomes the constraint: object storage, or a single large file.