LFM2.5-2.6B-DSpark-ROCmFPX-GGUF

LFM2.5-2.6B-DSpark — DSpark draft, GGUF

GGUF conversion of the DSpark speculator for LiquidAI/LFM2.5-2.6B, for use with llama.cpp --spec-type draft-dspark. Paired target (ROCmFP4): kingjones777/LFM2.5-2.6B-ROCmFP4-COHERENT-GGUF.

⛔ READ THIS FIRST — measured SLOWER on AMD Strix Halo

We benchmarked this draft against its target and it reduced throughput. Published for completeness and reproducibility, not as a recommended speedup.

Mean accepted length 2.58 (block size 9). AMD Ryzen AI Max+ 395 / Radeon 8060S (gfx1151), ROCm 7.13.0, idle box, median of 3, -ngl 999 -ngld 999 -fa on -c 2048 -n 512 --temp 0 --seed 1234. All three LFM2.5 sizes measured negative: −28.4% (1.2B), −19.1% (2.6B), −38.0% (8B-A1B).

Why — two runtime causes, not the weights

1. lfm2.cpp / lfm2moe.cpp do not populate t_layer_inp[], so draft-dspark aborts on GGML_ASSERT(t_layer_inp[il] != nullptr) out of the box. One-line fix: res->t_layer_inp[il] = prev_cur;
2. With that applied, llama.cpp reports recurrent state rollback is not compatible with 'draft-dspark' and falls back to a checkpoint path that is not bit-exact for LFM2's recurrent state — DSpark output diverges from greedy target output (reproducible 3/3).

An off-by-one in target-layer mapping was ruled out: LLAMA_DFLASH_TARGET_LAYER_OFFSET=-1 gave a worse accepted length (2.22), confirming the converter's +1 convention is correct.

On other stacks the same technique wins: DSpark measured +75% on Ling-3.0-flash under SGLang (22.3 → 39.1 t/s) on the same day. Speculative value belongs to the (model, runtime) pair.

Files

No 4-bit draft is published: the draft's entire value is accepted length, and 4-bit would lower it further to save ~400 MB.

Requirements

llama.cpp with DSpark support (--spec-type draft-dspark), plus the t_layer_inp fix above for LFM2 targets. Converted with upstream llama.cpp e85caa81ea2b65797396018c179b87ad61fa38ab.

llama-cli -m <target>.gguf -md LFM2.5-2.6B-DSpark-F16.gguf \
  --spec-type draft-dspark --spec-draft-n-max 8 \
  -ngl 999 -ngld 999 -fa on

Provenance

Converted from LiquidAI/LFM2.5-2.6B-DSpark at revision 458cedab07d0f7b2b05700c77e1aa463d43d6f04. Licence inherited from the base model.