Qwen3-4B-Dolci-Think-SFT-7B

Qwen3-4B-Dolci-Think-SFT-7B

Qwen/Qwen3-4B after full-parameter supervised fine-tuning on a 600k-example subset of allenai/Dolci-Think-SFT-7B.

The model emits reasoning inside <think>...</think> blocks, as the base model and the SFT data both do. It was trained as the starting checkpoint for research on compressing chain-of-thought traces into short sequences of abstract-token embeddings, but nothing about the checkpoint is specific to that use — it is an ordinary reasoning SFT of Qwen3-4B.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "sagisch/Qwen3-4B-Dolci-Think-SFT-7B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype="auto", device_map="auto")

messages = [{"role": "user", "content": "How many r's are in strawberry?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

out = model.generate(inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95, top_k=20)
print(tokenizer.decode(out[0][inputs.shape[-1]:], skip_special_tokens=True))

Use the Qwen3 thinking-mode sampling settings — temperature 0.6, top-p 0.95, top-k 20. Greedy decoding is explicitly discouraged for Qwen3 and tends to produce repetition collapse. Context length is 40,960 tokens (max_position_embeddings), inherited from the base model.

Training

Held-out eval loss over training (20k-example split, 512 examples per eval):

Train loss started at 1.2502 (step 10) and ended near 0.77–0.85. The curve is flat over the last ~1,000 steps, so this is a converged single epoch rather than an early stop.

These weights are the run's best checkpoint by held-out loss, cast to bf16. Optimizer state is not included — this checkpoint is for inference and further fine-tuning from scratch-initialized optimizer state, not for resuming the original run.

Evaluation

No benchmark numbers are reported here yet. Loss above is held-out cross-entropy on the Dolci eval split, which is a language-modeling metric, not a task-accuracy metric — do not read it as a capability comparison against the base model.

Limitations

• Single epoch on one SFT dataset, with no preference tuning or RL on top. It inherits the base model's limitations and adds Dolci-Think's distributional biases.
• No safety tuning was performed beyond whatever the base model and dataset carry.
• Not evaluated on standard reasoning benchmarks, so any claim of improvement over Qwen/Qwen3-4B is currently unsupported.

License

Apache-2.0, inherited from Qwen/Qwen3-4B. Dolci-Think-SFT-7B carries its own terms; see the dataset card.