Skip to main content

Module webgpu_lora

Module webgpu_lora 

Source
Expand description

GPU-accelerated LoRA delta application via wgpu compute shader.

Computes output += B @ (A @ x) * scaling entirely on the GPU, avoiding a CPU round-trip for the hidden-state delta.

§Bind-group layout (group 0)

bindingtypecontent
0storage rwoutput array (n_out)
1storage rinput array (n_in)
2storage rlora_a array (rank × n_in)
3storage rlora_b array (n_out × rank)
4uniformLoraGpuParams

Structs§

LoRAGpuApplicator
Holds the compiled wgpu pipeline for LoRA delta computation.
LoraGpuParams