A model can appear to generalize while recognizing examples it has effectively seen before. Randomly splitting rows does not prevent that when several rows belong to the same person, device, experiment, or document.
The first question is not which splitter to call. It is what future situation the evaluation should represent.
Rows versus independent entities
Suppose a dataset contains 1,000 measurements from 50 machines. Each machine contributes 20 measurements and has a characteristic sensor offset. A random row split will usually put measurements from the same machine in both training and test sets.
A model may use the offset as a shortcut. Its score then describes additional measurements from familiar machines more closely than performance on a new machine.
If deployment means new machines, assign every row from one machine to the same fold. The model must then transfer across entities rather than relying on their repeated signatures.
Fit preprocessing inside each split
Grouping does not fix preprocessing leakage. Imputation values, scaling statistics, feature selection, and dimensionality reduction must be learned from the training portion only.
A pipeline can keep those operations together with the estimator. During cross-validation, the training fold fits the preprocessing and model; the validation fold only receives the resulting transformations and predictions.
A feature built from the full dataset can leak even if the final estimator is inside a pipeline. Inspect feature construction as well as the split code.
Time is a separate constraint
If the task predicts future observations, preserve chronology. Grouping alone can still train on future measurements and evaluate on earlier ones. Conversely, a time split may retain familiar entities when the intended task is transfer to new entities.
Sometimes both constraints matter. State the deployment question explicitly: future data from existing machines, data from unseen machines, or future data from unseen machines. Each requires a different evaluation design.
Inspect the split, not just the score
Count unique entities in each fold and check that the intended intersections are empty. Inspect date ranges and label availability. Verify that duplicated records, derived windows, or near-identical documents do not cross the boundary under different identifiers.
Report the number of independent entities as well as the number of rows. Ten thousand rows from five machines do not provide the same evidence as ten thousand independent machines.
Interpret a lower score usefully
A grouped score may drop sharply. That can be a better estimate of the actual task rather than a worse model. Use the result to identify missing variation, unstable features, and the need for more representative data.
See scikit-learn’s common pitfalls and model-selection documentation. Ask Data Science Chat about grouped validation.