Qwimi3.5-9B-Kimik2.6-Opus-Distill-MTP-BF16

Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16

<p align="center"> <img src="./qwimi_3.5_9b_launch_overview.png" alt="Qwimi3.5-9B launch overview" width="100%"> </p>

Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16 is a merged BF16 fine-tuned version of unsloth/Qwen3.5-9B.

This model was trained as a reasoning-focused SFT/distillation run using a curated mixture of Kimi K2.6, Qwen reasoning, and Claude Opus TraceInversion-style reasoning datasets. The goal of this run was to improve structured reasoning behavior while preserving Qwen-style chat formatting and <think>...</think> reasoning traces.

• Developed by: trjxter
• Base model: unsloth/Qwen3.5-9B
• Model type: Merged BF16 causal language model
• Training method: LoRA / RS-LoRA SFT with Unsloth + TRL
• License: Apache 2.0
• Language: English

---

Training Overview

This model was trained with Unsloth and Hugging Face TRL using a LoRA-based supervised fine-tuning setup.

Training Configuration

Final Training Metrics

The lightweight eval loss was measured on a 200-example eval subset during training.

---

Dataset Mix and Curation

This run used a combined reasoning/distillation dataset made from three sources:

1. trjxter/Kimi-K2.6-Reasoning-3300x-WandB
2. Jackrong/Qwen3.5-reasoning-700x
3. Jackrong/Claude-opus-4.6-TraceInversion-9000x

The datasets were normalized into Qwen chat format and assistant reasoning traces were preserved in the following style:

<think>
...
</think>
final answer

After formatting and 16k token filtering, the final usable dataset contained 12,366 examples.

Dataset Credits

Special thanks to Jackrong and Kyle Hessling for the Opus and Qwen reasoning datasets used in this run. These datasets are not mine, and this model card acknowledges their work and dataset contributions.

The Kimi dataset used in this run was prepared under trjxter/Kimi-K2.6-Reasoning-3300x-WandB, while the Qwen reasoning and Claude Opus TraceInversion datasets came from Jackrong's public dataset releases.

---

Intended Use

This model is intended for experimentation with:

• reasoning-style SFT
• synthetic distillation
• long-context reasoning behavior
• math and structured problem solving
• coding and technical reasoning prompts
• local inference after conversion to quantized formats

This is an experimental fine-tune and should be evaluated carefully before use in production or high-stakes settings.

---

Prompt Format

The model follows Qwen-style chat formatting.

Example:

<|im_start|>user
Solve the problem step by step: If a worker earns $9/hour for 8 hours, plus $15 for each of 7 racquets, $10 for each of 2 grommet replacements, and $1 for each of 5 stencils, how much do they earn?
<|im_end|>
<|im_start|>assistant
<think>
...
</think>
...
<|im_end|>

When using transformers, prefer applying the tokenizer chat template instead of manually formatting prompts.

---

Loading with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-BF16"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

messages = [
    {
        "role": "user",
        "content": "Solve this step by step: A shop earns $72 from hourly pay, $105 from restringing, $20 from grommets, and $5 from stencils. What is the total?"
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=1024,
    temperature=0.6,
    top_p=0.95,
    do_sample=True,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=False))

---

Related Releases

This run may also be released in adapter and GGUF formats:

• LoRA adapter: trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-LoRA
• GGUF quants: trjxter/Qwimi3.5-9B-Kimik2.6-Opus-Distill-GGUF

---

Notes

This model was trained using Unsloth for efficient fine-tuning and Hugging Face TRL for SFT training.

<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>

---

Disclaimer

This is an experimental research fine-tune. Outputs may contain mistakes, hallucinations, or incorrect reasoning. Always validate important outputs independently.