Downloads · 30 days
3
7% of all-time downloads
burnssa/gemma-2-2b-medical-judge-v4
gemma-2-2b-medical-judge-v4 is a text classification model from burnssa. Use it when you need a label for a piece of text. It is set up for peft. The card lists the license as mit.
Ablation/checkpoint version. Use v5 for deployment. v4 is published for reproducibility of the experimental progression that motivated v5.
Downloads · 30 days
3
7% of all-time downloads
All-time downloads
42
Public
Repo size
534 MB
Likes
0
Public
Click a slice to open those files.
.pt333 MB · 49%
From the Hugging Face model README
Ablation/checkpoint version. Use v5 for deployment. v4 is published for reproducibility of the experimental progression that motivated v5.
v4 was the first attempt to address the Sonnet-style fixation found in v3 by SAE feature attribution. v3 generalized cross-architecture in sensitivity (caught Mistral-poisoned content) but its low-drift detector leaned on Sonnet-specific style features (markdown punctuation, structured formatting) — diagnosed in the v3 SAE analysis.
v4 fix: added 400 aligned responses from
microsoft/Phi-3.5-mini-instruct (a different family with a different
default style). Training data became 3,200 (v3) + 400 (Phi-3) = 3,600
records.
| metric | v3 | v4 | v5 |
|---|---|---|---|
| Sonnet 4.5 trained mean drift | 8.5 | 16.0 | 2.28 |
| Sonnet 4.5 flag@50 | 1.6% | 1.6% | 0% |
| Mistral-7B-poisoned flag@75 | 84.4% | 81.2% | 93.8% |
| in-distribution val MAE | 15.59 | 14.59 | 10.69 |
| in-distribution val Spearman | 0.747 | 0.783 | 0.802 |
The Sonnet absolute mean drift rose 8.5 → 16.0 in v4 — predictable from reduced reliance on Sonnet-style shortcuts (predictions regress toward training-set mean for OOD aligned inputs). Practical thresholded metrics (flag@50, flag@75) were preserved.
A second SAE feature attribution pass on v4 confirmed the predicted mechanism: every top Sonnet-style negative feature got 5–15% weaker.
| feature | description | v3 r | v4 r | Δ |
|---|---|---|---|---|
| 8265 | Sonnet punctuation marks | −0.665 | −0.591 | −0.074 (11% weaker) |
| 8783 | Sonnet social-media style | −0.630 | −0.560 | −0.070 |
| 12184 | Sonnet sentence endings | −0.607 | −0.525 | −0.082 |
| 16269 | Sonnet metadata structure | −0.553 | −0.522 | −0.031 |
Content positive features (medical guidance, problem/solution rhetoric) were preserved or strengthened — explaining why Mistral cross-arch sensitivity was retained.
v5 added 5-style GPT-4o-mini aligned augmentation (terse, conversational, bulleted, academic, markdown_heavy) on the same 400 prompts → 2,000 more aligned records. The result decoupled style from content far more aggressively than v4's single-family-addition approach. v5 dominates v4 on every behavioral metric.
The full ablation arc is documented at
judge_distillation/RESULTS.md.
Same as v5. Replace the adapter path:
adapter = "burnssa/gemma-2-2b-medical-judge-v4"
@misc{burns2026judgev4,
author = {Burns, Scott},
title = {Gemma-2-2B Medical Judge v4 (Phi-3-augmented ablation)},
year = {2026},
url = {https://huggingface.co/burnssa/gemma-2-2b-medical-judge-v4},
}