Downloads · 30 days
200
26% of all-time downloads
alea-institute/kl3m-007-500m-step100000
kl3m-007-500m-step100000 is a text generation model from alea-institute. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A 500M parameter language model trained with conservative hyperparameters using the Muon optimizer and spectral clamping. This checkpoint represents 100,000 training steps (8.32B tokens).
Downloads · 30 days
200
26% of all-time downloads
All-time downloads
780
Public
Parameters
500M
2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2 GB · 99%
From the Hugging Face model README
A 500M parameter language model trained with conservative hyperparameters using the Muon optimizer and spectral clamping. This checkpoint represents 100,000 training steps (8.32B tokens).
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"alea-institute/kl3m-007-500m-step100000",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("alea-institute/kl3m-007-500m-step100000")
inputs = tokenizer("The contract specifies", return_tensors="pt", return_token_type_ids=False)
inputs = {k: v.to(model.device) for k, v in inputs.items()}
outputs = model.generate(**inputs, max_new_tokens=100, temperature=0.8, top_p=0.95)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Apache 2.0