Downloads · 30 days
66
1% of all-time downloads
hiebo/Qwen3.6-34B-80L-Fable-5-Heretic
Qwen3.6-34B-80L-Fable-5-Heretic is a image-text-to-text model from hiebo. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product.
Fable-5 distilled Qwen3.6 80-layer deep reasoning model.
Downloads · 30 days
66
1% of all-time downloads
All-time downloads
4.7K
Public
Parameters
33.4B
67.8 GB on disk
Likes
9
Public
Click a slice to open those files.
.safetensors67.7 GB · 100%
From the Hugging Face model README
Fable-5 distilled Qwen3.6 80-layer deep reasoning model.
Built on the Qwen3.6-27B (64-layer, original) base, extended to 80 layers (~34B parameters) and distilled from Fable-5 Agentic Traces. Fine-tuned via QLoRA (4-bit NF4, rank=64) on 4,665 Fable-5 Chain-of-Thought trajectories, then merged with MTP speculative decoding heads.
| Property | Value |
|---|---|
| Base model | Qwen3.6-27B (64-layer, original) → Qwen3.6-34B-80L-BF16-v2 |
| Architecture | Qwen3_5ForConditionalGeneration |
| Parameters | ~34B (80 layers, hidden=5120) |
| Attention heads | 24 (4 KV Grouped) |
| Context length | 262,144 (256K) |
| Vocabulary size | 248,320 |
| Attention mode | Hybrid Linear/Full (full every 4 layers) |
| MTP speculative decoding | ✅ (15 mtp weights) |
| Precision | BF16 |
| Item | Value |
|---|---|
| Data source | Glint-Research/fable-5-traces |
| Samples | 4,665 entries |
| Content | Fable-5 Agentic CoT trajectories (context + completion) |
| Training config | QLoRA rank=64, α=64, 2 epochs, lr=2e⁻⁴, max_len=4096 |
q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, and MoE projection layersfrom transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "hiebo/Qwen3.6-34B-80L-Fable-5-Heretic"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype="auto",
device_map="auto",
)
prompt = "What is quantum computing?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
vllm serve hiebo/Qwen3.6-34B-80L-Fable-5-Heretic \
--dtype bfloat16 \
--max-model-len 262144 \
--enforce-eager \
--gpu-memory-utilization 0.95 \
--swap-space 16 \
--disable-log-stats
| Parameter | Default | Description |
|---|---|---|
--model | (required) | Model ID or local path |
--dtype | auto | Data type: bfloat16, float16, float32, auto |
--max-model-len | 4096 | Maximum context length (supports 256K) |
--max-num-seqs | 256 | Maximum number of concurrent sequences |
--gpu-memory-utilization | 0.90 | Fraction of GPU memory to use (0–1) |
--swap-space | 4 | CPU swap space in GB |
--tensor-parallel-size | 1 | Tensor parallelism degree (multi-GPU) |
--pipeline-parallel-size | 1 | Pipeline parallelism degree |
--enforce-eager | False | Force PyTorch eager mode (recommended for BF16 models) |
--disable-log-stats | False | Disable statistics logging |
--served-model-name | None | Override served model name |
--port | 8000 | API server port |
--host | 0.0.0.0 | API server host |
--tokenizer-mode | auto | Tokenizer mode: auto, slow, mistral |
--quantization | None | Quantization: fp8, awq, gptq, squeezellm |
--load-format | auto | Load format: auto, sharded_state, safetensors, npcache |
--kv-cache-dtype | auto | KV cache data type: auto, float16, bfloat16 |
--enable-chunked-prefill | False | Enable chunked prefill for long sequences |
--max-num-batched-tokens | None | Max batched tokens per step |
--seed | 0 | Random seed for reproducibility |
--distributed-executor-backend | ray | Distributed backend: ray, mp, mp_rank |
--trust-tokenizers | False | Trust remote tokenizer files |
--revision | main | Model revision (branch/tag/commit) |