Test set's split structure determines the claim about the model

The test set says: “Here’s what the model is evaluated on”, or answers the question: “Which claim are you making?”. Using random split? Interpolation. Using temporal split? Forecasting. Grouping by unit? Generalizing to new units.

The test set, unlike the validation set, is used post-training to evaluate the trained model on previously unseen data. It’s the final test of the models performance before it goes into production.