Qwen3-1.7B-Distilled-30B-A3B-GGUF

Qwen3-1.7B-Distilled-30B-A3B GGUF

GGUF quantizations of reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B — a 1.7B-parameter Qwen3 causal language model distilled from Qwen3-30B-A3B-Instruct using discrepancy-informed proof-weighted knowledge distillation on 6,122 STEM chain-of-thought samples.

Quantized by tinyopsec.

---

Model Details

---

About the Original Model

Qwen3-1.7B-Distilled-30B-A3B is trained with Discrepancy-Informed Knowledge Distillation (DISC v3) — a method that goes beyond standard KL divergence by treating per-token teacher-student divergence as a structured signal:

• Discrepancy-Weighted KD — identifies reasoning pivot tokens via local KL jump detection and amplifies their distillation weight
• DG-Limit Smoothing — stabilizes high-entropy student tokens with neighborhood averaging before KD is applied
• Gap Energy Regularization — monitors and penalizes structural drift at reasoning transitions independent of mean token loss
• Proof-Weighted Cross-Entropy — applies 2.5× → 1.5× decaying weight on derivation spans (Proof: to Final Answer:)

The model is particularly suited for STEM reasoning, mathematical derivations, and proof-style explanation.

---

Available Quantizations

---

VRAM Requirements

---

Usage

llama.cpp

./llama-cli -m model_q4_k_m.gguf \
  -p "Solve the following problem carefully and show a rigorous derivation.\n\nProblem:\nFind the eigenvalues of [[2,1],[1,2]].\n\nProof:\n" \
  -n 512 --temp 0.0

llama-cpp-python

from llama_cpp import Llama

llm = Llama(model_path="model_q4_k_m.gguf", n_ctx=2048)

output = llm(
    "Solve the following problem carefully and show a rigorous derivation.\n\nProblem:\nProve that sqrt(2) is irrational.\n\nProof:\n",
    max_tokens=512,
    temperature=0.0,
)
print(output["choices"][0]["text"])

LM Studio

1. Download any .gguf file from this repository
2. Open LM Studio → Load Model → select the file
3. Use the prompt format below in the chat or playground

Ollama

ollama run hf.co/tinyopsec/Qwen3-1.7B-Distilled-30B-A3B-GGUF

---

Prompt Format

For best results, use the training format:

Solve the following problem carefully and show a rigorous derivation.

Problem:
{your problem here}

Proof:

For general instruction use, standard chat template also works:

<|im_start|>user
{your question}<|im_end|>
<|im_start|>assistant

---

Intended Uses

• Mathematical derivations and proof-style explanation
• STEM problem solving (physics, engineering, linear algebra, differential equations)
• Educational tutoring and worked solutions
• Lightweight reasoning on edge or CPU-only hardware
• Generator component in verifier-generator or RAG reasoning pipelines

---

Limitations

• Not a formal proof verifier or symbolic algebra engine
• Can produce fluent but incorrect derivations
• Training context is 1024 tokens — very long derivations may degrade
• Domain coverage is uneven: physics and linear algebra are well-represented; molecular biology and physiology are sparse

---

Original Model

reaperdoesntknow/Qwen3-1.7B-Distilled-30B-A3B

Full methodology: Structure Over Scale (DOI: 10.57967/hf/8165)