Expand description
Residual Vector Quantizer (RVQ) — the codec structure used by discrete audio token
models (EnCodec / SoundStream / DAC family). This is pure codebook math: no learned weights
are required, so it is real and buildable now. The caller supplies the codebooks (each stage
a [n_centroids × dim] row-major block); the neural encoder/decoder that produces those
codebooks and the latent vectors is a separate, NeedsWeights concern (out of scope here).
Encoding is greedy per stage: at stage s we pick the centroid nearest (squared L2) to the
running residual — the input minus the centroids already chosen at stages 0..s. Decoding
sums the chosen centroids back. Zero-heap: no scratch is allocated; the running residual is
recomputed on the fly from the already-chosen tokens (stage/dim counts are small).
Functions§
- rvq_
dequantize - Decode
n_stagesresidual token indices back into adim-vector by summing the chosen centroids across all stages. - rvq_
quantize - Encode one
dim-vector inton_stagesresidual token indices.