BIZNIQUE / LINEAR ALGEBRA / 14
When scaling leaves a decision unchanged
When scaling leaves a decision unchanged
Distinguish a model’s mathematical invariance from its implementation details.
BIZNIQUE / LINEAR ALGEBRA / 14
Distinguish a model’s mathematical invariance from its implementation details.
THE IDEA
A positive affine change of one feature preserves its order. An exact threshold split can move its threshold by the same transformation and keep the same partition. This explains why ordinary decision-tree splits are often insensitive to feature units.
x ≤ t ⇔ ax + b ≤ at + b, for a > 0
This is not a universal exemption from preprocessing. Approximate binning, numerical precision, regularisation and other parts of a pipeline may behave differently.
TRY IT YOURSELF
A split uses x ≤ 10. If z = 2x + 3, what threshold preserves the split?
Use z ≤ 23.