A permutation-invariant representation guarantees that the feature vector depends only on content and never on order
An identical set of rows should always yield the same feature vector. If relative to the target row-order is noise, by not following this rule the feature vector partially becomes a function of order - something that was just defined as noise.