gpt-oss-20b-transcript-formatter-lora

gpt-oss-20b Transcript Formatter — LoRA adapter (distilled student)

A LoRA adapter for openai/gpt-oss-20b that turns a raw speech-to-text transcript into a clean, formatted transcript — fixing punctuation, casing, fillers, disfluencies, homophones, ITN, garbled proper nouns / phonetic URLs-emails, and spoken layout commands — while preserving the speaker's words and passing through anything that is already correct.

This is the distilled 20B student of Akash-Sakala/gpt-oss-120b-transcript-formatter-lora. It reproduces the teacher's formatting behaviour at a fraction of the cost: direct final channel, no chain-of-thought, and a 20B base that fits on a single large-VRAM GPU.

This repo ships only the LoRA adapter (~hundreds of MB), not the merged 20B. Merge it on demand against the public base — see Usage.

Model details

Training approach — distillation

The student is trained by distilling the 120B teacher onto the 20B base (Phase-1 offline SFT):

1. Teacher generation. The RL-final 120B teacher produces gold formatted outputs over the transcript-formatter-curriculum prompts (all 21 categories, L0–L5 + RL-align).
2. Student SFT. gpt-oss-20b is fine-tuned (LoRA rank 32, lr 2e-4 cosine, 2 epochs, batch 64, max_len 4096, AdamW β=(0.9, 0.95)) to match the teacher's outputs on the direct final channel — the reasoning trace is dropped, so the student answers immediately without chain-of-thought.
3. Combined corpus. All curriculum layers are pooled into a single 18,505 (input → gold) pair SFT mix (priority-ordered so every category and the passthrough negative-traps are represented), rather than trained stage by stage. Layer composition:

21 categories total, 18,505 pairs. Full dataset: Akash-Sakala/transcript-formatter-curriculum.

Results

Held-out evaluation over 2,384 rows across all 21 categories, scored with the same two-round protocol as the teacher: deterministic Round 1 (EM / ContentOK / CER), then an independent Claude judge re-adjudicates Round-1 fails to credit cases where the model is correct but the gold label is unrealistic ("adjusted accuracy").

Student numbers measured over 2,384 held-out rows (all 21 categories), Round-1 judge claude-sonnet-4-6; 44 Round-1 fails re-adjudicated → 33 rescued, 11 confirmed fails. Source: final_eval_all_layers_20B_p1.json / final_eval_round1_20B_p1.json. The distillation hits teacher parity (99.5% vs 99.9% adjusted acc) at 20B with no reasoning — cheaper and faster to serve, and the student's EM/CER/WER slightly beat the teacher.

EM understates quality for the layout categories (many valid paragraph/list renderings); read ContentOK / adjusted accuracy / CER, not EM.

Usage

⚠ The system prompt is required (the model was trained with it verbatim).

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained(
    "openai/gpt-oss-20b", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base, "Akash-Sakala/gpt-oss-20b-transcript-formatter-lora")
tok = AutoTokenizer.from_pretrained("openai/gpt-oss-20b")

SYSTEM_PROMPT = "..."  # the verbatim training system prompt
messages = [{"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": "i spoke to aisha she confirmed the PR will be merged by friday"}]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(ids, max_new_tokens=4096, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=False))
# → "I spoke to Aisha. She confirmed the PR will be merged by Friday."

Merge once for serving (vLLM/SGLang):

# Option A — PEFT
merged = PeftModel.from_pretrained(base, "Akash-Sakala/gpt-oss-20b-transcript-formatter-lora").merge_and_unload()
merged.save_pretrained("./gpt-oss-20b-transcript-formatter-merged")

# Option B — Tinker cookbook
from tinker_cookbook import weights
weights.build_hf_model(
    base_model="openai/gpt-oss-20b",
    adapter_path="./adapter",            # this repo, downloaded locally
    output_path="./merged",
    dtype="bfloat16",
)

Hardware: gpt-oss-20b is a 20B MoE — far cheaper than the 120B teacher. Greedy decoding (temperature 0) for deterministic output.

Intended use & limitations

• Use: post-processing ASR/Whisper output into readable text (dictation, meeting notes, voice memos, emails).
• Preserves words. It edits format, not meaning — it does not paraphrase, summarise, answer, or invent content; ambiguous cases pass through unchanged.
• Limitations: English only; rare/novel proper-noun spellings can be missed; paragraph segmentation of free-dictated emails is subjective; not a general assistant (it only formats transcripts).
• License: Apache-2.0, inherited from the base openai/gpt-oss-20b.

Citation / provenance

Distilled with the Tinker API from teacher gpt-oss-120b-transcript-formatter-lora. Student base openai/gpt-oss-20b, LoRA rank 32, lr 2e-4 cosine, 2 epochs, batch 64, direct-final (no reasoning), renderer gpt_oss_no_sysprompt. Phase-1 distillation run. Tinker checkpoint: ac4a6812-1d65-507a-9f93-080fab31f7e4:train:0/sampler_weights/final. See the dataset card for the full curriculum.