Downloads · 30 days
10
1% of all-time downloads
abhishekai/gemma-2-2b-legal-raft-v2
gemma-2-2b-legal-raft-v2 is a text generation model from abhishekai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as gemma.
Gemma-2-2B trained with RAFT (Retrieval-Augmented Fine-Tuning) on top of abhishekai/gemma-2-2b-legal-sft-v2, using QLoRA 4-bit.
Downloads · 30 days
10
1% of all-time downloads
All-time downloads
667
Public
Parameters
2.6B
5.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors5.2 GB · 99%
From the Hugging Face model README
Gemma-2-2B trained with RAFT (Retrieval-Augmented Fine-Tuning) on top of
abhishekai/gemma-2-2b-legal-sft-v2,
using QLoRA 4-bit.
RAFT trains one specific competence: read three documents, answer from the one that contains the answer — or decline when none of them does. That is the thing this model does better than its parent, and essentially the only thing.
On ordinary grounded QA, where the correct passage is always supplied, it is indistinguishable from its parent:
| eval | n | raft-v2 | sft-v2 | diff | 95% CI |
|---|---|---|---|---|---|
| grounded QA (passage always correct) | 294 | 8.80 | 8.98 | −0.14 | [−0.32, +0.04] |
On the RAFT eval — three documents per question, sometimes none of them sufficient — the pooled number is also flat, and the pooled number is misleading:
| condition | n | raft-v2 | sft-v2 | diff | 95% CI | |
|---|---|---|---|---|---|---|
| all prompts | 299 | 5.81 | 5.65 | +0.16 | [−0.11, +0.45] | no change |
| answerable | 259 | 5.72 | 5.88 | −0.16 | [−0.35, +0.03] | no change |
| unanswerable | 75 | 6.06 | 3.83 | +2.23 | [+1.03, +3.40] | improves |
The overall +0.16 is a large gain on 13% of rows averaged against a small loss on the other 87%.
The unanswerable subgroup was judged in two passes: 40 rows in the first 300-prompt sample, then the remaining 35 separately. The second pass reproduced the first — +2.22 on rows the model had never been scored on, against +2.24 — which is why the pooled interval tightens rather than moves. The sign test, which disagreed at n=40, agrees at n=75: p = 0.023, 41 W / 22 L / 12 T.
The first two rows stay on the balanced 300-prompt sample on purpose. Pooling the top-up into them would make the judged set 22.5% unanswerable against this eval's real 12.5%, and turn the flat overall row into a spurious +0.38 that excludes zero.
Counting refusal phrasings directly in the generations, no model-as-judge involved:
| correct declines (of 75 unanswerable) | false declines (of 260 answerable) | |
|---|---|---|
sft-v2 | 23 (31%) | 21 (8%) |
raft-v2 | 41 (55%) | 10 (4%) |
It declines more often when it should and less often when it should not — discrimination improving, rather than a refusal threshold sliding in one direction. This matters because a model that simply refuses more would look identical on the unanswerable rows alone.
sft-v2's 23 declines are 23 distinct sentences, each written from the passage in front
of it. The decision is right; the phrasing is memorised, and a retrieval system that wants a
reason for the refusal will not get one.The caveat that did not survive: at n=40 the sign test disagreed with the interval (p = 0.229). With the subgroup complete it agrees — p = 0.023 over 63 decisive pairs.
QLoRA 4-bit, LoRA r=16 on all attention and MLP projections (20.8M trainable, 1.28%). One epoch, 2,087 steps, 1×H100, 72 minutes, $4.76.
Training data is 33,398 rows: each question paired with an oracle document plus a partial distractor (same source, does not contain the answer) and an irrelevant one (different source), at a 12.5% unanswerable share — mixed 50/50 with a general-instruction distribution to prevent forgetting, which two earlier narrow fine-tunes of this model did not survive.
Best validation loss 0.9294, at the final step: it did not converge, so this is a floor for the recipe rather than its ceiling. It is the only stage in this project to complete its learning-rate schedule.
Reading the pooled validation loss alone would have made this run look dead — it moved 0.0119 while the grounded half of the split moved 0.064.
Gemma-2's chat template, with the system instruction folded into the user turn (Gemma-2 raises on
a system role). The assistant turn ends on <end_of_turn> (107), not <eos> — generation
must stop on it. Supply the candidate documents in the user turn as context.
Claude Sonnet as an independent judge — a different model family from the Gemini 2.5 Flash that generated the training data. Four axes out of 10 (question answering, instruction following, grounding, appropriate refusal), scored against the prompt, the answer, a golden answer and the corpus evidence. Both models were generated on the same prompts with the same seed, and scored in the same run, so the comparison is paired throughout.
gemma-2-2b-legal-sft-v2 instead.