Downloads · 30 days
0
peluz/qwen3-0.6b-cat-lingo-grpo
qwen3-0.6b-cat-lingo-grpo is a machine learning model from peluz. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A Qwen3-0.6B model fine-tuned in two stages:
Downloads · 30 days
0
Access
Public
Updated Jun 16, 2026
Repo size
335 MB
Likes
0
Public
Click a slice to open those files.
.safetensors323 MB · 63%
From the Hugging Face model README
A Qwen3-0.6B model fine-tuned in two stages:
| Item | Value |
|---|---|
| Base model | Qwen/Qwen3-0.6B |
| Stage 1 | SFT (TRL) + LoRA (r=32), 10 epochs |
| Stage 2 | GRPO (TRL) + LoRA continued, 3 epochs |
| GRPO generations | 4 per prompt |
| Reward | Single holistic LLM judge (cat authenticity + factual quality) |
| Judge | Qwen2.5-0.5B-Instruct (4-bit NF4) |
| β (KL penalty) | 0.04 |
| GPU budget | 8 GB |
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
base = AutoModelForCausalLM.from_pretrained(
"Qwen/Qwen3-0.6B", torch_dtype=torch.bfloat16, trust_remote_code=True
)
model = PeftModel.from_pretrained(base, "peluz/qwen3-0.6b-cat-lingo-grpo")
tokenizer = AutoTokenizer.from_pretrained("peluz/qwen3-0.6b-cat-lingo-grpo", trust_remote_code=True)
messages = [{"role": "user", "content": "Explain black holes."}]
text = tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
)
inputs = tokenizer([text], return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))
Trained on a compact dataset; cat persona may occasionally drop on unusual prompts.