Expand description
GPU-accelerated LoRA delta application via wgpu compute shader.
Computes output += B @ (A @ x) * scaling entirely on the GPU,
avoiding a CPU round-trip for the hidden-state delta.
§Bind-group layout (group 0)
| binding | type | content |
|---|---|---|
| 0 | storage rw | output array<f32> (n_out) |
| 1 | storage r | input array<f32> (n_in) |
| 2 | storage r | lora_a array<f32> (rank × n_in) |
| 3 | storage r | lora_b array<f32> (n_out × rank) |
| 4 | uniform | LoraGpuParams |
Structs§
- LoRA
GpuApplicator - Holds the compiled wgpu pipeline for LoRA delta computation.
- Lora
GpuParams