Downloads · 30 days
20
3% of all-time downloads
slm-125m-base
karthiksab/slm-125m-base
slm-125m-base is a text generation model from karthiksab. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
125.8M-parameter decoder-only language model trained from scratch for legal, financial, and educational English text.
All-time downloads
771
Public
Parameters
126M
252 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors252 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Jul 10, 2026
- Updated
- Jul 10, 2026
- SHA
- 31c9a0b0
Try a prompt
Model card
From the Hugging Face model README
slm-125m-base
125.8M-parameter decoder-only language model trained from scratch for legal, financial, and educational English text.
Training
- Architecture: 12-layer Llama-compatible decoder, hidden size 768, vocabulary 16,384
- Context length: 1,024 tokens
- Training data: 2.034B packed training tokens; 20.6M validation tokens
- Data mix: cleaned and deduplicated US case law, SEC filings, and FineWeb-Edu
- Optimization: AdamW, BF16, cosine learning-rate schedule, one epoch
- Hardware: 8 NVIDIA H100 GPUs
- Final validation loss: 2.3981
- Final validation perplexity: 11.00
- Reported GPU compute cost: approximately $10.60
This is a base model, not an instruction-tuned or safety-tuned assistant. Outputs may be inaccurate and should not be treated as legal or financial advice.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Jul 10, 2026
- Updated
- Jul 10, 2026