gemma4-31b-cessna-dpo-fp8

gemma4-31b-cessna-dpo

A Gemma 4 (31B) assistant for the Cessna 172S, aligned with DPO to answer in a clean, professional, pilot-friendly style. It is the preference-tuned successor to the supervised model Aurumdev95/gemma4-31b-cessna, whose knowledge was distilled from the Cessna 172S Pilot's Operating Handbook (POH).

⚠️ Study aid only. This model is for flight-training and familiarization. It is not an authoritative reference and must not be used for real-world flight operations, planning, or decision-making. Always consult the official POH and current approved documentation.

What DPO changed

The supervised model already answered POH questions accurately. DPO was applied to fix presentation: correct technical spelling, adopt consistent terminology, and format answers as well-structured Markdown (short bold labels, ordered steps, bullet lists) so they render cleanly and read the way a briefing would.

• Preference data: 133 train / 15 validation pairs. For each question the SFT model's own answer was the rejected response; an expert-editor rewrite that preserved every technical fact while improving clarity and formatting was the chosen response.
• Objective: DPO (sigmoid) with an rpo_alpha = 1.0 SFT anchor to prevent chosen-logprob collapse; 3 epochs, beta = 0.1, LoRA r = 16.

Results

Held-out preference accuracy rose each epoch — the model learned to prefer the professional, well-formatted answers:

Crucially, factual accuracy did not regress. On the same 46-question held-out POH validation set (served in NVFP4 via vLLM, greedy decoding):

The lexical-overlap scores are essentially unchanged (the DPO answers are longer and more structured, which these terse-reference metrics do not reward), while the style is markedly more professional — the intended outcome.

Variants

Prompting

Use a flight-assistant system prompt, e.g.:

You are a knowledgeable flight assistant for the Cessna 172S. Answer questions about the aircraft's systems, limitations, and procedures accurately and concisely, based on the Pilot's Operating Handbook.

Training stack

Unsloth + TRL 0.24 + Transformers 5.5 on an NVIDIA DGX Spark (GB10). See the training pipeline for full configuration.