BIZNIQUE / CALCULUS / 53
Trace sensitivity through time
Trace sensitivity through time
Unroll a simple recurrence and follow how an earlier state affects a later one.
Embedded Files
THE IDEA
Read the example.
Read the example.
For this scalar recurrence, the influence of the initial state is a product of local derivatives. Backpropagation through time applies the chain rule through an unrolled computational graph, with shared parameters receiving contributions from multiple steps.
hₜ = ahₜ₋₁; ∂hₜ/∂h₀ = aᵀ
Use it well
Use it well
The demo shows sensitivity to the initial state, not the full derivative with respect to a shared parameter. Truncating the history changes which dependencies contribute to an update.
TRY IT YOURSELF
A quick check.
A quick check.
With a = 1 and no other terms, how does the sensitivity to h₀ change as T increases?
Check your reasoning
Check your reasoning
It remains 1 in this idealised recurrence.
Page updated
Google Sites
Report abuse