ViAble-1.7-9B

ViAble-1.7-9B

ViAble-1.7-9B is a reasoning-focused finetune of Qwen 3.5 9B (qwen3_5 architecture): supervised finetuning on ~98k distillation traces, followed by a GRPO phase. This repo holds the merged bf16 weights (no adapter needed).

Requires transformers>=5.15.1 (the version that adds qwen3_5 support). Always load with trust_remote_code=True.

Base model

• Base: Qwen 3.5 9B — 9B params, 32 layers (dense hybrid attention, qwen3_5 architecture).

Training

• SFT: 1 epoch on 98,455 distillation samples (r0b0tlab/qwen3.8-max-glm5.2-kimi-k3-distillation, canonical), LoRA r=64 / alpha=128, seq 2048, bf16, lr 2e-4 cosine, effective batch 16 (~6,154 steps).
• GRPO: continued training (~3,000 steps, 4 generations/prompt, beta 0.04, lr 5e-6).
• Hardware: Kaggle GPUs (RTX 6000 Pro for training).

Benchmarks

Eval protocol (both runs): chat template from tokenizer_config.json, system prompt asks for step-by-step thinking; stage 1 lets the model think (up to 1,024 tokens), stage 2 forces the final answer (The correct answer is X / \boxed{...}) for samples that did not conclude.

Resource note — please read before comparing

These runs were done on limited free-tier compute (Kaggle T4 / RTX 6000 Pro), so:

• MMLU here is scored generatively (the model writes the answer letter), not with the loglikelihood setup used by harnesses such as lighteval — numbers are not directly comparable across harnesses.
• HLE is a frontier benchmark where even the largest models score modestly; 4.44% is in the expected range for a 9B model. Image-based items (342/2500) were answered from text only.

If you have more compute, please reproduce or extend full-set evals with the protocol above and share results via a discussion — thank you!

Usage

pip install -U "transformers>=5.15.1" accelerate

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "bogairff55/ViAble-1.7-9B"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True,
)

msgs = [{"role": "user", "content": "15 + 27 = ?"}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
x = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**x, max_new_tokens=512)
print(tok.decode(out[0][x.input_ids.shape[1]:], skip_special_tokens=True))

Limitations

• Reasoning traces can be long; set max_new_tokens to fit your budget.
• As with all LLMs, outputs may contain errors or hallucinations; verify for critical use.