Downloads · 30 days
7
1% of all-time downloads
Covanent-Base-Non-Chat
diagonalge/Covanent-Base-Non-Chat
Covanent-Base-Non-Chat is a machine learning model from diagonalge. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Covenant72B is the largest permissionless collaboratively trained language model trained entirely from scratch at the 72 billion parameter scale.
All-time downloads
653
Public
Parameters
72.7B
291 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors291 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Oct 30, 2025
- Updated
- Oct 30, 2025
- SHA
- 497725c8
Model card
From the Hugging Face model README
Covenant72B
Covenant72B is the largest permissionless collaboratively trained language model trained entirely from scratch at the 72 billion parameter scale.
It is being trained with 20+ globally distributed participants coordinated via decentralized infrastructure on the Bittensor blockchain.
Checkpoint-One marks the first release, corresponding to 200 billion tokens processed. Model files are available in the Checkpoint-One branch. Future checkpoints will be updated here.
Training Details
| Property | Value |
|---|---|
| Model size | 72B |
| Architecture | LLaMA-style |
| Target token budget | 1.2T (210B for current checkpoint) |
| Compute participants | 20+ |
| Minimal compute per participant | 8×B200 or equivalent |
| Dataset | DCLM-baseline |
| Optimizer | SparseLoCo (communication-efficient optimizer) |
Performance on Benchmarks
All results are 0-shot acc-norm (%)
| Model | Compute Environment / Permissions | Size | Tokens | ARC-C | ARC-E | PIQA | OpenBookQA | HellaSwag | Winogrande | MMLU |
|---|---|---|---|---|---|---|---|---|---|---|
| Intellect-1 | Over the internet / White List | 10B | 1T | 44.8 | 71.6 | 77.7 | 43.6 | 70.5 | 63.1 | 32.7 |
| Psyche Consilience-7Y9 | Over the internet / White List | 40B | 1.2T | 31.1 | 55.8 | 76.1 | 34.8 | 63.7 | 57.0 | 24.2 |
| Covenant72B – Checkpoint One | Over the internet / Permissionless | 70B | 210B | 46.2 | 72.6 | 79.2 | 43.0 | 73.5 | 70.3 | 38.0 |
| K2 Checkpoint 54 | Centralized Cluster | 65B | 210B | 41.8 | 69.5 | 80.1 | 42.4 | 74.9 | 68.9 | 33.7 |
For more details, refer to Checkpoint One on Templar Research.
- Type
- llama
- License
- apache-2.0
- Created
- Oct 30, 2025
- Updated
- Oct 30, 2025