Ling-3.0-flash-DSpark-GGUF

Ling-3.0-flash DSpark GGUF

GGUF quantizations of inclusionAI DSpark draft model for Ling-3.0-flash.

Use with BeeLlama.cpp, a llama.cpp fork with advanced quantization features.

---

Ling3-DSpark

A DSpark speculator for Ling3. DSpark extends DFlash with target-model auxiliary features and a confidence head that dynamically chooses the number of draft tokens. The model was trained with SpecForge and is served with SGLang.

Model specifications

• Target model: Ling-3.0-flash
• Draft parameters: 1,363,707,905 (1.36B)
• Draft weight dtype: BF16
• Hidden size: 2,560
• Transformer layers: 5 full-attention layers
• Attention: MHA with 32 query heads and 32 key/value heads
• Target auxiliary feature layers: 1, 11, 23, 29, 35
• Confidence head: vanilla Markov head, rank 256
• DSpark block size: 8 draft tokens (verify width 9, including the target bonus token)
• Maximum position embeddings: 262,144

Acceptance length

Acceptance length is the mean number of tokens accepted per speculative verification step, including the target bonus token.

The macro mean across the nine workload means is 5.29.

Serving with SGLang

Launch recipes for this draft on every supported hardware/quantization cell — including the required --linear-replayssm-cache-len sizing — with measured speed and accuracy, are in the SGLang Ling-3.0-flash cookbook.

Use an SGLang version with DSPARK support. Replace the model paths and tensor-parallel size with values appropriate for your deployment:

sglang serve \
  --trust-remote-code \
  --model-path <LING3_MODEL_PATH> \
  --tp-size <TP_SIZE> \
  --speculative-algorithm DSPARK \
  --speculative-draft-model-path <LING3_DSPARK_MODEL_PATH> \
  ......

Serving with llama.cpp

Use a llama.cpp build with DSpark support. Replace the model paths, quantization type, and GPU layer counts with values appropriate for your deployment. First convert and quantize the target model:

python convert_hf_to_gguf.py path/to/Ling-3.0-flash \
    --outfile path/to/Ling-3.0-flash-bf16.gguf \
    --outtype bf16 --model-name Ling-3.0-flash

llama-quantize \
    path/to/Ling-3.0-flash-bf16.gguf \
    path/to/Ling-3.0-flash-Q4_K_M.gguf \
    Q4_K_M

Then generate the DSpark draft GGUF:

python convert_hf_to_gguf.py \
    path/to/Ling-3.0-flash-dspark \
    --target-model-dir path/to/Ling-3.0-flash \
    --outtype bf16 \
    --outfile path/to/Ling-3.0-flash-DSpark.gguf

Finally, launch the server with the DSpark draft as the speculative model:

llama-server \
    --model path/to/Ling-3.0-flash-Q4_K_M.gguf \
    --spec-draft-model path/to/Ling-3.0-flash-DSpark.gguf \
    --spec-type draft-dspark --spec-draft-n-max 8 \
    -ngl all -ngld all -fa on