Downloads · 30 days
26
3% of all-time downloads
tvall43/Qwen3.5-2B-Claude-4.6-Opus-Reasoning-Distilled-heretic-v1
Qwen3.5-2B-Claude-4.6-Opus-Reasoning-Distilled-heretic-v1 is a text generation model from tvall43. Use it when you need the model to write or continue text. The card lists the license as apache-2.0.
giving up here, trial 19 was pretty good. for some reason i havent bothered to figure out, trails in heretic are taking over 20 minutes with this model specifically. a full 200 trial run would take over 70 hours. mpoa…
Downloads · 30 days
26
3% of all-time downloads
All-time downloads
879
Public
Parameters
2.2B
4.4 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors4.4 GB · 100%
From the Hugging Face model README
giving up here, trial 19 was pretty good. for some reason i havent bothered to figure out, trails in heretic are taking over 20 minutes with this model specifically. a full 200 trial run would take over 70 hours. mpoa+soma with just attn.out_proj like ive had good luck with abliterating the rest of the qwen3.5 models.
| Parameter | Value |
|---|---|
| direction_index | 9.31 |
| attn.o_proj.max_weights.0 | 0: 0.93 |
| attn.o_proj.max_weights.1 | 1: 1.04 |
| attn.o_proj.max_weights.2 | 2: 0.84 |
| attn.o_proj.max_weights.3 | 3: 1.21 |
| attn.o_proj.max_weight_position | 15.56 |
| attn.o_proj.min_weights.0 | 0: 0.21 |
| attn.o_proj.min_weights.1 | 1: 0.54 |
| attn.o_proj.min_weights.2 | 2: 0.11 |
| attn.o_proj.min_weights.3 | 3: 0.78 |
| attn.o_proj.min_weight_distance | 10.95 |
| Metric | This model | Original model (Jackrong/Qwen3.5-2B-Claude-4.6-Opus-Reasoning-Distilled) |
|---|---|---|
| KL divergence | 0.0086 | 0 (by definition) |
| Refusals | 5/100 | 91/100 |
Update: This model has been further enhanced with additional reasoning data distilled from Qwen3.5-27B.
The new training data introduces higher-quality reasoning trajectories across domains such as science, instruction-following, and mathematics.
Part of the data comes from Jackrong/Qwen3.5-reasoning-700x, a curated dataset designed to improve structured step-by-step reasoning and reasoning diversity.

Qwen3.5-2B-Claude-4.6-Opus-Reasoning-Distilled is a highly capable reasoning model fine-tuned on top of the Qwen3.5-2B dense architecture. The model's core directive is to leverage state-of-the-art Chain-of-Thought (CoT) distillation primarily sourced from Claude-4.6 Opus interactions.
Through Supervised Fine-Tuning (SFT) focusing specifically on structured reasoning logic, this model excels in breaking down complex user problems, planning step-by-step methodologies within strictly formatted <think> tags, and ultimately delivering precise, nuanced solutions.
Base Model (Qwen3.5-2B)
│
▼
Supervised Fine-Tuning (SFT) + LoRA
(Response-Only Training masked on "<|im_start|>assistant\n<think>")
│
▼
Final Model Text Only (Qwen3.5-2B-Claude-4.6-Opus-Reasoning-Distilled)
The model includes targeted optimizations addressing Qwen3.5’s tendency toward excessive transitional or repetitive reasoning on simple queries. Through deep distillation and structural imitation of Claude-4.6-Opus reasoning chains, the model adopts a more efficient structured thinking pattern:
“Let me analyze this request carefully: 1..2..3...”.
This streamlined reasoning paradigm significantly reduces redundant cognitive loops while preserving deep analytical capacity, resulting in substantially improved inference efficiency.
Let me analyze this request carefully:
1. Identify the core objective of the problem.
2. Break the task into clearly defined subcomponents.
3. Evaluate constraints and edge cases.
4. Formulate a step-by-step solution plan.
5. Execute the reasoning sequentially and verify consistency.
.
.
.
train_on_responses_only strategy, masking instructions so the loss is purely calculated over the generation of the <think> sequences and the subsequent solutions.<think> {internal reasoning} </think>\n {final answer}.The training loss showed a strong and healthy downward trend throughout the entire 3-epoch run, demonstrating effective knowledge distillation. Starting from an initial loss of 0.730115, the model converged steadily to a final loss of 0.186790 — indicating the model successfully internalized the structured <think> reasoning patterns from the Claude 4.6 Opus teacher data.
The dataset consists of high-quality, filtered reasoning distillation data:
| Dataset Name | Description / Purpose |
|---|---|
| nohurry/Opus-4.6-Reasoning-3000x-filtered | Provides comprehensive Claude 4.6 Opus reasoning trajectories. |
| TeichAI/claude-4.5-opus-high-reasoning-250x | Injecting high-intensity, structured reasoning instances. |
| Jackrong/Qwen3.5-reasoning-700x | Additional curated reasoning samples designed to strengthen structured step-by-step problem solving and improve reasoning diversity. |
<think> block sequentially rather than exploratory "trial-and-error" self-doubt.Significant thanks to the Unsloth AI team for making rapid fine-tuning of large LLM models accessible. Additionally, we acknowledge Qwen internally, and the open-source community developers producing exceptional distilled datasets (nohurry and TeichAI).