Grouped cross-validation
Grouped cross-validation keeps observations from the same entity together in one fold. It tests transfer to unseen entities rather than recognition of familiar ones.
Definition
Grouped cross-validation keeps observations from the same entity together in one fold. It tests transfer to unseen entities rather than recognition of familiar ones.
Intuition
Rows are not always independent. Repeated observations from one person or device can leak identity across a random split.
Worked example
If one customer has 20 purchases, place all 20 in the same fold when evaluating performance on new customers.
The math
Use a group identifier to partition samples. For future prediction, also respect time; grouping alone does not prevent temporal leakage.
In practice
Use grouped evaluation for users, machines, experiments, or subjects with repeated measurements.
Go deeper
- GuideCross-Validation for Small ML Datasets
- InteractiveDecision Tree Explorer
Sources
More in Data science
Assembled from the ReLU.chat curated knowledge base. These explanations are concise on purpose; check the sources for anything important.