Condition number
A condition number measures how sensitive a problem solution is to input perturbations. It describes the problem, distinct from algorithmic stability.
Definition
A condition number measures how sensitive a problem solution is to input perturbations. It describes the problem, distinct from algorithmic stability.
Intuition
A stable algorithm can still produce a sensitive answer when the underlying problem is ill-conditioned.
Worked example
diag(1, 0.001) has 2-norm condition number 1000, so some small perturbations can be strongly amplified.
The math
For invertible A, .
In machine learning
Inspect conditioning before blaming precision alone. Scaling, regularization, or changing the formulation may help.
Go deeper
Sources
More in Linear algebra
Assembled from the ReLU.chat curated knowledge base. These explanations are concise on purpose; check the sources for anything important.