keys-Qwen3.8-27B-DSpark-vLLM-YaRN-1M-Context

Qwen3.8-27B-DSpark — vLLM + YaRN 1M

Config-only fork of Doopeworld/Qwen3.8-27B-DSpark-vLLM (= byte-identical RadixArk/Qwen3.8-27B-DSpark weights).

Stock DSpark is native 262,144. 1M on Qwen3.8 is YaRN factor 4 on the target. SGLang/vLLM then copy that override into the draft; the draft is a flat Qwen3 config, so YaRN either crashes (text_config / max_position_embeddings) or the draft rope stays at 262k while the target is at 1M.

This checkpoint does for DSpark what Keys already did for DFlash2 (dflash2_config_yarn_1m.json): put the 1M YaRN rope on the draft.

What changed (weights untouched)

- "max_position_embeddings": 262144
+ "max_position_embeddings": 1048576
  "rope_parameters": {
-   "factor": 32.0,
-   "original_max_position_embeddings": 8192,
+   "factor": 4.0,
+   "original_max_position_embeddings": 262144,
    "rope_type": "yarn",
    "rope_theta": 10000000
  }

Architectures stay Qwen3DSparkModel so vLLM does not route this to DeepSeek-V4.

Serve (vLLM)

Bind this config.json over the draft. Keep the target YaRN hf-overrides. Do not also inject nested text_config into the draft.

--max-model-len 1048576 \
--hf-overrides '{"text_config":{"max_position_embeddings":1048576,"rope_parameters":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144,"rope_theta":10000000}}}' \
--speculative-config '{"method":"dspark","model":"/drafter","num_speculative_tokens":7,"draft_sample_method":"probabilistic"}'

This is a rope overlay, not a retrain. Acceptance at 1M is unmeasured until a 1M target + this draft is benched.

Credit

Weights and DSpark method: RadixArk / SpecForge. vLLM architecture rename: Doopeworld. YaRN-1M draft config: Keys (drowzeys).