Lingbot-VA-W4A8

LingBot-VA — W4A8 SVDQuant

4-bit weight / 8-bit activation post-training quantization of LingBot-VA (RoboTwin post-trained checkpoint), produced with an SVDQuant-style pipeline: per-channel SmoothQuant, a rank-32 FP16 low-rank branch, and a group-wise INT4 residual.

This repository contains the real quantized payload — packed INT4 codes — not a "fake-quant" checkpoint that stores dequantized FP16 weights.

Contents

What is quantized

The 300 transformer block Linear layers of the LingBot-VA video-action DiT, matched by (^|\.)blocks\.\d+\.. Embeddings, heads, the VAE and the text encoder are left in their original precision.

params quantized     4.91 B
packed INT4          2340 MB
aux (fp16)            305 MB   scales + rank-32 branch + smoothing vector + bias
effective            4.52 bits / param
on disk              2.58 GB   (vs 9.82 GB at bf16 — 3.8x smaller)

Tensor format

Per layer:

INT4 codes are stored as unsigned nibbles offset by +8 (stored 0..15, true range -8..7); the low nibble is the even column.

Dequantization

lo = (qweight & 0x0F).to(torch.int16) - 8      # even columns
hi = ((qweight >> 4) & 0x0F).to(torch.int16) - 8  # odd columns
codes[:, 0::2], codes[:, 1::2] = lo, hi

W = (codes.reshape(out, in_f // 64, 64) * wscale.unsqueeze(-1)).reshape(out, in_f)
W = W + lr_b @ lr_a                            # smoothed weight
y = linear(quant_a8(x * inv_smooth), W) + bias

W is the smoothed weight, so the input must be scaled by inv_smooth before the matmul — the two are inverse halves of one transform and separating them changes the layer's function. Activations are quantized at runtime to INT8, per-token, group-64.

Calibration

50 RoboTwin episodes — one randomly sampled per task across all 50 tasks, seed 42 — using all 10,919 frames for the per-input-channel activation statistics.

Verification

Payload integrity. All 300 layers present with advertised shapes/dtypes; INT4 codes within [-8, 7]; pack/unpack round-trips losslessly; dequantized weights reproduce the smoothed FP weights to a worst-case relative error of 4.85e-04 (fp16 rounding from applying the low-rank branch to the output rather than folding it into W).

Closed-loop behaviour. Run in RoboTwin simulation with the quantized weights actually driving the policy:

These are 10-episode functional checks confirming the checkpoint behaves correctly end-to-end — they are not a benchmark success rate, and should not be quoted as one.

Performance note

Dequantizing per forward and running an ordinary GEMM is slower than bf16 (measured 2.8–6.9× slower on these layer shapes), because rebuilding the weight costs more than the reduced memory traffic saves. Realising a speedup requires a fused kernel that never materializes the weight. Note that fused SVDQuant kernels such as Nunchaku implement W4A4, not W4A8, so they cannot serve this checkpoint directly.

License

Derivative of LingBot-VA and inherits its licensing terms. Consult the base model's license before use or redistribution.