BIZNIQUE / LINEAR ALGEBRA / 05
Read a batch training step
Read a batch training step
Follow predictions, residuals and a gradient without hiding them behind a training loop.
Embedded Files
THE IDEA
Read the example.
Read the example.
For half the mean squared error, the residual vector is Xw − y. Multiplication by Xᵀ aggregates each feature’s contribution to those residuals. A batch gradient uses the chosen batch of observations to propose one weight update.
∇L(w) = Xᵀ(Xw − y) / n
Use it well
Use it well
Faster arithmetic does not guarantee better generalisation. Keep preprocessing, data splits, objective definition and stopping conditions consistent when comparing implementations.
TRY IT YOURSELF
A quick check.
A quick check.
Why does changing the loss from half-MSE to MSE double this gradient?
Check your reasoning
Check your reasoning
MSE omits the factor 1/2, so differentiating the squared error leaves a factor of 2.
Page updated
Google Sites
Report abuse