Downloads · 30 days
7
17% of all-time downloads
slm-125m-base-5ep
thesreedath/slm-125m-base-5ep
slm-125m-base-5ep is a machine learning model from thesreedath. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
A 125M-parameter Llama-style base language model, pretrained from scratch on a legal/financial corpus (US case law + SEC filings + a web slice, ~2.19B tokens).
All-time downloads
42
Public
Parameters
126M
252 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors252 MB · 100%
At a glance
- License
- other
- Model type
- llama
- Access
- Public
- Created
- Jul 11, 2026
- Updated
- Jul 11, 2026
- SHA
- 06658545
Model card
From the Hugging Face model README
slm-125m-base-5ep
A 125M-parameter Llama-style base language model, pretrained from scratch on a legal/financial corpus (US case law + SEC filings + a web slice, ~2.19B tokens).
- Pretraining: 5 epochs, 8xH100. Held-out validation perplexity: 8.5.
- This is a base COMPLETER, not a chat model. Prompt it with the start of a sentence and it continues in the legal register. It speaks the register fluently but does not know facts (knowledge is capped at ~2 bits/param at this size).
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("thesreedath/slm-125m-base-5ep")
model = AutoModelForCausalLM.from_pretrained("thesreedath/slm-125m-base-5ep")
ids = tok("The plaintiff alleges that the defendant", return_tensors="pt").input_ids
print(tok.decode(model.generate(ids, max_new_tokens=60, do_sample=True,
temperature=0.8, min_new_tokens=40)[0]))
Full pipeline + fine-tuning code: https://github.com/Vizuara-AI-Lab/slm-125m-from-scratch
- Type
- llama
- License
- other
- Languages
- en
- Created
- Jul 11, 2026
- Updated
- Jul 11, 2026