Ornith-1.5-9B-DFlash-GGUF

Ornith 1.5 9B DFlash GGUF

A target-specific classic DFlash draft for ornith-ai/Ornith-1.5-9B, initialized from z-lab/Qwen3.5-9B-DFlash and distilled against Ornith 1.5 target features.

This is not a standalone language model. Pair it with the Ornith 1.5 9B target in a runtime that supports classic DFlash speculative decoding.

File

Training

The six-layer Qwen3.5 DFlash backbone was retained. Projection-specific distillation unfroze fc and hidden_norm, while the remaining draft layers stayed frozen.

• Target: ornith-ai/Ornith-1.5-9B
• Initialization: z-lab/Qwen3.5-9B-DFlash
• Corpus: 144 target-generated Alpaca prompts and responses
• Target features: cached from the BF16 Ornith target
• Training: 4,096 projection-only optimization steps
• Objective: position-weighted block cross-entropy following the DFlash objective
• Evaluation mean accepted length: 2.50 before → 2.77 after distillation
• Export: GGUF Q4_K_M using current llama.cpp tooling

The training pipeline is included in this repository as dflash_distill_mlx.py.

DS4 usage

./ds4 \
  -m Ornith-1.5-9B-Q4_K_M.gguf \
  --dflash ornith1.5-9b-dflash-bf16-projection-Q4_K_M.gguf \
  -p "Count from 1 to 100."

DS4's adaptive scheduler is enabled by default. It probes draft yield, increases the speculative block only when profitable, and falls back to target-only decoding on low-yield text. Set DS4_DFLASH_ADAPTIVE=0 to disable adaptation.

Measured results

Apple M4, Ornith 1.5 9B Q4_K_M target, greedy decoding:

The counting workload improved by 16.8%. Low-yield text does not benefit; its short benchmark includes the one-time probe cost, after which DS4 stays target-only.

Greedy target-only and speculative outputs were byte-identical for a normal chat prompt and a 128-token counting conversation.

Limitations

• Target-specific: acceptance may be substantially lower with base Qwen3.5 or another fine-tune.
• Throughput depends on runtime kernels, hardware, prompt distribution, and accepted length.
• The reported exactness checks cover greedy decoding in DS4. Other runtimes and sampling modes were not qualified here.
• This repository contains only the draft. Download the target separately from ornith-ai/Ornith-1.5-9B-GGUF.

Acknowledgements and licenses

The draft initialization is from Z-Lab's Apache-2.0 Qwen3.5 DFlash release. Ornith 1.5 is released under MIT. See the upstream repositories for their respective license texts and model documentation.