Columnar format
A file format that stores values grouped by column rather than by row, so a reader can fetch only the fields it needs.
On a wide table this is often the single largest reduction in read volume available, which matters when I/O rather than compute is the bottleneck.
Data & I/OPerformance & optimization
Related terms
- I/O-bound: A workload whose runtime is limited by reading and writing data rather than by calculation. Adding workers to an I/O-bound job adds waiting, not throughput.
- Partition: One independently processable chunk of a dataset. Partition size is the main lever on how well a workload parallelizes.