Reproducibility is the ceteris-paribus condition that makes replication-variance a clean measurement

The pipeline must run deterministically across identical runs, so that we’re sure that when we update a single axis, the run satisfies ceteris paribus. Once that condition is satisfied, replication-variance experiment can measure the variance of noise using replication knobs.

see A replication knob is a source of variation under a fixed experimental design for information about a replication knob see Run variation sources the axes along which ceteris paribus can be violated for the axes that must satisfy reproducibility