Pawel Flajszer Slip-box Notebooks Work About

written: 21 Apr 2026
rests on: 3
cited by: 3

Validation and Test sets must have identical split structure to support the same claim

Discrepancy between the two would choose a wrong model for the claim, or the wrong claim for the model. Because of the role the validation and test sets have, they both must match in the context of their split structure.

Rests on

The split type has to preserve the structure you're generalizing along general principle this is an instance of.
Test set's split structure determines the claim about the model the role of the test set in model training.
Validation set's split structure selects the model the role of the validation set in model training.

Cited by

Test set's split structure determines the claim about the model the dependency on the validation dataset.
The split type has to preserve the structure you're generalizing along a constraint when splitting
Validation set's split structure selects the model the dependency on the validation dataset.