gemma-2-2b-legal-sft

gemma-2-2b-legal-sft

A grounded legal reader: it answers a question using only a passage you supply, and refuses when the passage does not contain the answer. QLoRA fine-tune of google/gemma-2-2b-it — 4-bit NF4 frozen base plus 16-rank LoRA adapters, merged back to bf16 for release.

It exists as the controlled counterpart to narendraalluri/slm-125m-sft, a 125M model trained from random weights on the same records. Both were trained on the same 11,563 examples and evaluated on the same 608 held-out questions — the split was replayed and verified token-for-token against the 125M run's tokenised artifact (608/608 rows matched), which is what licenses the side-by-side table below.

Gemma Terms of Use

This is a derivative of Gemma and is governed by the Gemma Terms of Use and the Gemma Prohibited Use Policy. By using it you agree to those terms.

Prompt format (required)

Trained on one exact template — Gemma 2 has no system role, so the instruction folds into the user turn. Deviating from this degrades grounding:

<bos><start_of_turn>user
Answer the question using only the provided context. If the context does not contain the answer, say so.

<context>
{passage}
</context>

Question: {question}<end_of_turn>
<start_of_turn>model

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("narendraalluri/gemma-2-2b-legal-sft")
model = AutoModelForCausalLM.from_pretrained("narendraalluri/gemma-2-2b-legal-sft")

SYSTEM = "Answer the question using only the provided context. If the context does not contain the answer, say so."
prompt = (f"<bos><start_of_turn>user\n{SYSTEM}\n\n<context>\n{passage}\n</context>\n\n"
          f"Question: {question}<end_of_turn>\n<start_of_turn>model\n")

ids = tok(prompt, return_tensors="pt", add_special_tokens=False).input_ids
out = model.generate(ids, max_new_tokens=64, min_new_tokens=4, do_sample=False)
print(tok.decode(out[0][ids.shape[1]:], skip_special_tokens=True))

Greedy decoding is recommended. When the answer is absent the model emits exactly: The provided context does not contain the answer to that question.

Measured behaviour

On 58 unanswerable and 75 answerable held-out questions, greedy decoding. All three columns are the same questions:

The zero-shot column is the control that matters: refusal recall is scored on the exact refusal string, which an untuned instruct model has no reason to produce, so read it alongside the phrasing-insensitive variant — 60.3% zero-shot versus 98.3% after QLoRA. What the fine-tune teaches is the contract (one fixed refusal string, answers confined to the passage); how much of the underlying reading ability was already there is what the two Gemma columns separate.

Training

Everything up to and including <start_of_turn>model\n is masked to -100, so the model is never trained to reproduce the passage or the question — only to answer.

Data provenance and synthetic-data disclosure

The instruction data is synthetic. Passages were sampled from a cleaned corpus of US case law, SEC filings and general web text; the questions and answers were generated by Google Gemini (gemini-3.6-flash) and then filtered: every number in an answer must appear in its passage, answers must overlap the passage by >=60% of content words, plus exact dedup and 13-gram decontamination.

This model is the product of sequence-level distillation of a capability (grounded reading), though not of the teacher's knowledge, since the filter discards facts absent from the passage.

Limitations

• Not an assistant. It answers questions about a supplied passage. With no <context> it has nothing to read.
• No retrieval. Supplying the right passage is the caller's job; pair it with a retriever for RAG.
• Corpus decontamination gap. The passage corpus was screened against held-out legal benchmarks, but a measured share of documents matched n-grams that an earlier screening bug had skipped. No benchmark score should be quoted for this model without re-decontaminating first.
• Base-model behaviour, biases and limitations carry over from google/gemma-2-2b-it.
• Not legal or financial advice. This is a teaching demonstration of QLoRA on a pretrained base, not a production tool.