Data Leakage
Data leakage occurs when information from the test set sneaks into the training process, making evaluation overly optimistic.
Definition
Leakage means your model sees the answer during training — either directly (test features in training) or indirectly (future information encoded in features).
Common forms: using the target as an input feature, normalizing before splitting (test statistics influence training), duplicate rows across splits.
Intuition
A model with leakage has a misleadingly high evaluation — in production on truly new data, it will perform much worse.
The key principle: any transformation applied to the full dataset before splitting leaks information from the test set into training.
Worked example
Including "has_defaulted_next_month" as a feature is direct leakage — by definition it contains future information.
Computing mean encoding on the combined train+test set lets test-set means influence training features.
The math
Proper pipeline: split first, then fit any statistics (mean, std, encoder) on training data only, then transform both train and test.
For target encoding, use only the training fold's target means — fold-wise encoding avoids leakage in CV.
In practice
Always audit features for leakage before training; ask "would I have this feature at prediction time in production?"
If a feature's importance disappears after removing leakage, it wasn't a real signal.
Go deeper
More in Data science
Assembled from the ReLU.chat curated knowledge base. These explanations are concise on purpose; check the sources for anything important.