A replication knob is a source of variation under a fixed experimental design

Given a set of parameters like a random seed, weight initialisation in a Neural Network, or a choice of K-Folds, a replication knob would be dialling one of them, keeping the rest fixed across experiment runs. Comparing across those runs allows us to measure the noise variance.

see A replication knob must be both an arbitrary choice and unrelated to the target variable for its properties