Downloads · 30 days
784
52% of all-time downloads
alea-institute/kl3m-006-170m-checkpoint-57000
kl3m-006-170m-checkpoint-57000 is a text generation model from alea-institute. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A 170M parameter language model trained on multi-domain legal text using the Muon optimizer with spectral clamping. This checkpoint represents Phase 2 multi-domain training following Phase 1 single-domain training (37…
Downloads · 30 days
784
52% of all-time downloads
All-time downloads
1.5K
Public
Parameters
182M
739 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors727 MB · 98%
From the Hugging Face model README
A 170M parameter language model trained on multi-domain legal text using the Muon optimizer with spectral clamping. This checkpoint represents Phase 2 multi-domain training following Phase 1 single-domain training (37K steps).
alea-institute/kl3m-data-sample-004-balancedAnalysis of weight matrix conditioning shows acceptable manifold quality with expected degradation from relaxed regularization:
Key Finding: Attention layers show higher condition numbers than Phase 1 due to intentionally relaxed spectral regularization. This trade-off enables faster multi-domain learning and exploration while maintaining numerical stability.
| Metric | Phase 1 (37K) | Phase 2 (57K) | Change |
|---|---|---|---|
| Attention Conditioning | Median: 1620 | Median: 2307 | ↑ 42% |
| Generation Quality | 5.7/10 | 8.5/10 | ↑ 49% |
| Multi-Domain Coverage | EDGAR only | 4 domains | ✓ |
| Spectral Clamp Freq | Every 10 steps | Every 160 steps | 16x relaxed |
Result: Intentional conditioning trade-off yields significantly better generation quality and multi-domain capability.
Generates coherent, fluent legal text across multiple domains with no repetition issues. Significantly improved over Phase 1:
Suitable for multi-domain legal content generation, analysis, and understanding tasks.
from transformers import pipeline
# Create text generation pipeline
generator = pipeline(
"text-generation",
model="alea-institute/kl3m-006-170m-checkpoint-57000",
torch_dtype="auto",
device_map="auto"
)
# Generate text
outputs = generator(
"This Agreement is entered into as of",
max_new_tokens=100,
temperature=0.7,
top_p=0.95,
repetition_penalty=1.15
)
print(outputs[0]['generated_text'])
This checkpoint demonstrates the intentional conditioning vs quality trade-off:
The 16x reduction in spectral clamping frequency allows the model to explore richer multi-domain manifolds while maintaining acceptable numerical stability (max condition < 4000).
Alea Institute
For technical details, see the paper: https://arxiv.org/abs/2504.07854
@misc{kl3m2025,
title={KL3M: Knowledge-Guided Language Model Training},
author={Alea Institute},
year={2025},
url={https://arxiv.org/abs/2504.07854},
note={Trained with Muon optimizer and multi-domain spectral regularization}
}
Apache 2.0