tests
32
An optimisation ladder where every rung has to clear its own measured noise floor before it counts. Run on CPU, on a rented NVIDIA L4, and across two A40s over NCCL.
| rung | samples/s | spread | data-wait | vs prev | separable |
|---|---|---|---|---|---|
| baseline | 789.8 | 0.7 | 95% | ||
| workers | 3146.4 | 71.2 | 79% | +298% | yes |
| +persistent | 3003.6 | 23.7 | 80% | -4.5% | yes |
| +prefetch | 3072.8 | 20.4 | 79% | +2.3% | yes |
| +pin_memory | 3121.0 | 82.2 | 81% | +1.6% | no |
| +tf32 | 3174.7 | 83.6 | 81% | +1.7% | no |
| +amp (bf16) | 3682.7 | 655.2 | 79% | +16.0% | no |
| +channels_last | 3527.6 | 771.0 | 79% | -4.2% | no |
| +batch128 | 3644.1 | 666.5 | 88% | +3.3% | no |
| +compile | 3257.8 | 672.4 | 90% | -10.6% | no |
Separable means the change clears the spread of its own repeats. Six of nine rungs do not, and are reported as unestablished rather than quietly counted.
workers alone is +298%. Everything after it sums to about +17%, and only three of eight later rungs clear their own spread.
Data-wait never drops below 79%, starting at 95%. Compute got roughly four times faster on the L4 and the dataloader did not, which is why every kernel-level rung lands in the noise.
CPU pinning measured +41.3% on a laptop. On a quiet machine the noise floor fell from 11.8% to 1.0% and the effect became +0.6%. It had been contention all along.
On the L4 the cheapest route to target accuracy ranks 4th of 5 by throughput. On CPU it ranked first on both, so the CPU never exposed the disagreement.
The scaling study predicted its shape would transfer to NCCL and its numbers would not. Both held: throughput moved 51x, efficiency stayed in band, and the comm fraction went from noise to a quarter of the step.