Downloads · 30 days
14
6% of all-time downloads
abhishekai/slm-125m-legal-sft-v2
slm-125m-legal-sft-v2 is a text generation model from abhishekai. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A 125M-parameter Llama-architecture model, supervised fine-tuned from abhishekai/slm-125m-legal-base for grounded question answering over legal and financial passages.
Downloads · 30 days
14
6% of all-time downloads
All-time downloads
246
Public
Parameters
126M
503 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors503 MB · 100%
From the Hugging Face model README
A 125M-parameter Llama-architecture model, supervised fine-tuned from
abhishekai/slm-125m-legal-base
for grounded question answering over legal and financial passages.
v2 replaces slm-125m-legal-sft.
The dataset was rebuilt: 17,170 training pairs instead of 9,096, balanced so no single source
document dominates, with a deliberately harder held-out set.
| score /10 | |
|---|---|
| v1 SFT | 4.05 |
| v2 SFT | 4.13 |
Paired difference +0.09, 95% CI [−0.22, +0.39] — the interval contains zero, so no improvement is established. The same dataset moved the 500M by +0.39 and Gemma-2-2B by +0.92.
That is a capacity signal, and an unusually clean one: all three models saw identical data through identical code, and only the smallest failed to move. It also early-stopped after less than a third of one pass, where the 500M ran more than twice as far. Roughly 4.3% of its output collapses into repetition, so part of the ceiling is generation quality rather than comprehension.
v2 is published because it is the checkpoint the rest of the pipeline builds on, and because a negative result measured properly is worth more than an unmeasured positive one.
Q&A pairs written by Gemini 2.5 Flash over a legal/financial corpus, then filtered by a grounding self-check, embedding dedup, length and format gates, per-document balancing, and decontamination against the eval split.
3 epochs configured; early-stopped at step 1,000 of 3,183, best weights restored before saving. Validation loss 1.1415. 1×H100, $0.58.
The validation split is carved from the training data. Earlier runs validated on the judge's eval set, which meant selecting the checkpoint by looking at the exam — their published eval losses are optimistic by an unknown amount. This one is not.
It expects a chat template with a system prompt, and a user turn containing the question
followed by Context: and the passage. Use tokenizer.apply_chat_template.
Claude Sonnet as an independent judge — a different model family from the Gemini 2.5 Flash that wrote the training data, which breaks the circularity that makes a teacher-judged score partly self-congratulation. Four axes out of 10 (question answering, instruction following, grounding in the supplied passage, appropriate refusal), scored against the prompt, the answer, a golden answer and the corpus evidence. The same 300 held-out prompts score every model in this project, so the numbers are comparable across models — which win-rates against each model's own baseline never were.
The judge was itself gated before use: it had to reproduce a known ordering (base Gemma-2-2B above a broken fine-tune of it) before any of its scores were trusted. It did, at p = 0.0022.