Skip to main content

Module rvq

Module rvq 

Source
Expand description

Residual Vector Quantizer (RVQ) — the codec structure used by discrete audio token models (EnCodec / SoundStream / DAC family). This is pure codebook math: no learned weights are required, so it is real and buildable now. The caller supplies the codebooks (each stage a [n_centroids × dim] row-major block); the neural encoder/decoder that produces those codebooks and the latent vectors is a separate, NeedsWeights concern (out of scope here).

Encoding is greedy per stage: at stage s we pick the centroid nearest (squared L2) to the running residual — the input minus the centroids already chosen at stages 0..s. Decoding sums the chosen centroids back. Zero-heap: no scratch is allocated; the running residual is recomputed on the fly from the already-chosen tokens (stage/dim counts are small).

Functions§

rvq_dequantize
Decode n_stages residual token indices back into a dim-vector by summing the chosen centroids across all stages.
rvq_quantize
Encode one dim-vector into n_stages residual token indices.