Downloads · 30 days
10
10% of all-time downloads
sverma8873/slm-125m-legal-sft
slm-125m-legal-sft is a text generation model from sverma8873. Use it when you need the model to write or continue text. The card lists the license as other.
Instruction-tuned from [thesreedath/slm-125m-base] (https://huggingface.co/thesreedath/slm-125m-base) on synthetic grounded question-answer pairs over a US case-law / SEC-filings corpus.
Downloads · 30 days
10
10% of all-time downloads
All-time downloads
104
Public
Parameters
126M
252 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors252 MB · 100%
From the Hugging Face model README
Instruction-tuned from [thesreedath/slm-125m-base]
(https://huggingface.co/thesreedath/slm-125m-base) on synthetic grounded
question-answer pairs over a US case-law / SEC-filings corpus.
| metric | value |
|---|---|
| val loss | 1.8796 |
| val perplexity | 6.55 |
| epochs | 3 |
| optimizer steps | 261 |
| tokens seen | 4.28M |
| GPU | 1x A10G, ~2.5 min |
14,522 grounded QA pairs built with Gemini 2.5 Flash as teacher and
Gemini 3.1 Flash-Lite as judge, then filtered through a five-gate gauntlet:
format checks, context-leak detection, an LLM judge (grounding / correctness /
standalone, each >= 4/5), a lexical-overlap grounding backstop, embedding
near-duplicate clustering, and 13-gram decontamination against
casehold/casehold and coastalcph/lex_glue.
Yield was 72.6% of 19,991 raw pairs. Mean lexical overlap with the source passage is 0.885. Questions are spread near-uniformly across a 12-way archetype taxonomy (factual-lookup, definitional, procedural, causal, conditional, comparative, numerical, obligation, temporal, scope-limit, summarize-point, entity-role), with sources at 45% case-law / 35% SEC / 20% general web.
Uses the chat template shipped in tokenizer_config.json:
<|bos|><|system|>{system}<|eos|><|user|>{question}<|eos|><|assistant|>
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("sverma8873/slm-125m-legal-sft")
model = AutoModelForCausalLM.from_pretrained("sverma8873/slm-125m-legal-sft")
prompt = tok.apply_chat_template(
[{"role": "system", "content": "You are a legal and financial assistant. Answer using only the information you were given. If the answer is not available, say so."},
{"role": "user", "content": "What is a motion to dismiss?"}],
tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt")
out = model.generate(input_ids=ids["input_ids"],
attention_mask=ids["attention_mask"],
max_new_tokens=90, do_sample=True, temperature=0.7,
eos_token_id=tok.convert_tokens_to_ids("<|eos|>"))
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))
This model has no reliable factual recall. At 125M parameters it has essentially no capacity to store facts, and fine-tuning transferred format and style, not knowledge. It answers in fluent, appropriately-scoped legal prose and it will confidently invent case citations, dates, and statutory references.
It was trained closed-book: the source passage was not included in the prompt, so the model answers from parametric memory. Treat outputs as fluent drafts requiring verification, never as legal or financial advice.