gemma4-31b-rovochat-sft-v3

Gemma-4-31B Rovo Chat Orchestrator — Full SFT v3 (LR=2e-6)

Full-parameter SFT at LR=2e-6. Overall 76.97% — best tool accuracy (54.74%, beats base). Demonstrates that v2's collapse was an LR problem, not an inherent full-FT flaw.

Experiment: Gemma-4-31B SFT on Rovo Chat Orchestrator

Fine-tuning google/gemma-4-31B-it on the Rovo Chat Orchestrator dataset (6,602 multi-turn agentic traces). Evaluated on 725 held-out examples (100% text-gold) with an LLM judge (Claude Opus 4.8).

Results

Key Findings

• Learning rate was the dominant factor for full-FT. At LR=5e-6 (v2) the model catastrophically collapsed (47.72%, tool-loop timeouts → only 501/725 inferred). At LR=2e-6 (v3) it recovered to 76.97% with no loops (720/725 inferred).
• v3 tool accuracy (54.74%) beats the base model (53.16%) — full-FT at the right LR genuinely learns the tool-calling task. The residual gap to base is in text quality (84.86% vs 96.07%) — partial catastrophic forgetting.
• LoRA (r=16, frozen base) ≈ v3 (75.45% vs 76.97%), reached with far less compute (8 GPUs vs 40) and no LR sensitivity.

Technical Notes

• num_kv_shared_layers=0 for 31B dense — the Gemma-4 KV-shared/use_cache=False bug (transformers PR #45312) does NOT apply (E2B/E4B only).
• attn_implementation="sdpa" (global layers head_dim=512 > FlashAttention 256 cap). final_logit_softcapping=30 applied by the model forward.
• DeepSpeed ZeRO-3 (full-FT, 40×H200) / ZeRO-2 (LoRA, 8×H200), transformers 5.10.2, seq=16384–32768.

Inference

temperature=0.0, top_p=1.0, max_tokens=16000, enable_thinking=False