Feature Engineering

Feature engineering transforms raw data into model-friendly inputs, often determining model success more than algorithm choice.

Ask the Data science assistant1 min read · Updated September 9, 2026

Definition

Techniques: creating interaction terms ( $x_1 \cdot x_2$ ), polynomial features ( $x^2$ ), binning continuous variables, and encoding categoricals.

Date features: extract year, month, day, day-of-week, is_weekend, days_since_reference from a date column.

The model can only learn relationships that exist in the features — thoughtful feature construction gives the model something to work with.

Domain knowledge helps: for housing prices, "age at sale" (sale_year - build_year) is often more predictive than either alone.

For text: bag-of-words, TF-IDF, or word embeddings convert text to numeric vectors.

For timestamps: "hour_of_day" captures different behaviors for morning vs. evening; "is_holiday" captures event spikes.

One-hot encoding for low-cardinality categoricals: $K$ categories become $K-1$ binary columns (drop one to avoid multicollinearity).

Target encoding replaces a categorical with the mean of the target for that category — use carefully with cross-validation to avoid leakage.

Good feature engineering reduces the complexity needed from the model, improving interpretability and generalization.

Always create features on the training set and apply the same transformation to test/production data.

Assembled from the ReLU.chat curated knowledge base. These explanations are concise on purpose; check the sources for anything important.