Qwen3.6-27B-DrSparse-GRPO

Qwen3.6-27B — Dr.Sparse GRPO (LoRA adapter)

LoRA adapter fine-tuned on top of Qwen/Qwen3.6-27B with GRPO (Dr.GRPO variant) to improve reasoning-driven sparse-matrix kernel authoring for the Dr.Sparse project.

The trained model produces multi-turn plans and iteratively refines CUDA/cuSPARSE kernels; each rollout is graded by a compiled+profiled reward server (compute-sanitizer memcheck gate + cuSPARSE benchmark + optional Nsight Compute metrics).

Adapter details

Reward stack

• smooth_log reward curve
• best_plus_progress mode
• AUC weight 0.15, degrade penalty 0.15
• Verifier + planner turns included, verify-final gate, up to 5 attempts
• Optional NCU (Nsight Compute) profiling signal on GPU 3

Load

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3.6-27B"
adapter = "DiogenesChen122/Qwen3.6-27B-DrSparse-GRPO"

tok = AutoTokenizer.from_pretrained(base, trust_remote_code=True)
mdl = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16",
                                           device_map="auto", trust_remote_code=True)
mdl = PeftModel.from_pretrained(mdl, adapter)

Files

• adapter_config.json, adapter_model.safetensors — PEFT LoRA weights
• tokenizer.json, tokenizer_config.json, chat_template.jinja — tokenizer artifacts copied from base

Caveats

• Trained on a research corpus of sparse-matrix kernels (~472 curated prompts); domain generalization outside sparse linalg / CUDA is not tuned for.
• Chat template inherits from base — use apply_chat_template with add_generation_prompt=True.
• The reward server was not published — reproducing training requires the Dr.Sparse repo.