Downloads · 30 days
9
4% of all-time downloads
nyxia/Auron-510M
Auron-510M is a machine learning model from nyxia. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Downloads · 30 days
9
4% of all-time downloads
All-time downloads
225
Public
Parameters
744M
26.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.5 GB · 99%
From the Hugging Face model README

Auron — Chimera hybrid GDN-Attention language models with Ouroboros weight sharing.
Paper: Auron: Depth-Efficient Language Models via Hybrid Recurrent-Attention Weight Sharing Code: github.com/Fy-/Auron

git clone https://github.com/Fy-/Auron && cd Auron && rye sync
from ouro import load_model, generate
model, tokenizer, device = load_model("nyxia/Auron-510M")
generate(model, tokenizer, device, "The history of")
Default: T=0.7, top_k=20, top_p=0.9, rep_pen=1.0, presence_pen=1.5 (Ouroboros weight sharing requires presence penalty >= 1.5 to prevent attractor wells).
Built by Florian Gasquez (@nyxia). Part of the Soulkyn project.