BIZNIQUE / LINEAR ALGEBRA / 21
Compress a term–document representation
Compress a term–document representation
See how a rank-limited approximation can retain broad structure while losing detail.
Embedded Files
THE IDEA
Read the example.
Read the example.
A term–document matrix records how terms relate to documents. Truncated SVD can replace the original coordinates with a smaller set of latent directions. Our tiny numerical matrix illustrates reconstruction, not language understanding.
X ≈ UᵣΣᵣVᵣᵀ
Use it well
Use it well
Weighting, tokenisation and rank choice affect the representation. A latent direction is not guaranteed to correspond to a clean human topic, and truncation can discard rare but important information.
TRY IT YOURSELF
A quick check.
A quick check.
Does a smaller reconstruction error prove that a text representation is better for every task?
Check your reasoning
Check your reasoning
No. Reconstruction quality and downstream task performance measure different things.
Page updated
Google Sites
Report abuse