Downloads · 30 days
0
AGILLM-4.3-clean-20260802
MarxistLeninist/AGILLM-4.3-clean-20260802
AGILLM-4.3-clean-20260802 is a text generation model from MarxistLeninist. Use it when you need the model to write or continue text. The card lists the license as other.
Adaptive General Intelligence Large Language Model v4.3 — a 1.22B parameter language model with novel DiffusionBlock + Mixture-of-Experts architecture.
Access
Public
Updated Aug 2, 2026
Repo size
1.7 TB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.pt1.7 TB · 100%
At a glance
- Task
- Text Generation
- License
- other
- Access
- Public
- Created
- Aug 2, 2026
- Updated
- Aug 2, 2026
- SHA
- 5ecfcb1c
Try a prompt
Model card
From the Hugging Face model README
AGILLM 4.3
Adaptive General Intelligence Large Language Model v4.3 — a 1.22B parameter language model with novel DiffusionBlock + Mixture-of-Experts architecture.
Architecture
| Component | Details |
|---|---|
| Parameters | 1.22B (1,221,580,802) |
| Architecture | Transformer + DiffusionBlock + MoE |
| DiffusionBlock | 14 blocks, loss-balanced schedule, transformer router |
| MoE | 2 experts, top-1, shared expert (mult=2) |
| Attention | Sublinear (window=128, stride=128, sinks=4) |
| Context | 1536 tokens |
| Optimizer | AdamW 8-bit with cosine LR decay |
| Training Objective | Stochastic (AR 45% / SAT 25% / NAT 30%) |
Training Status
🟢 Actively training on RTX 3090 Ti (fedC recovery run)
- Progress: ~33% through 67.2B token run (~22.1B tokens seen)
- Throughput: ~70k tok/s
- Current step: ~897,000 (recovery from step 727,857)
- LR Schedule: Cosine decay (min_mult=0.10)
- Data: fineweb, fineweb-edu, wikipedia, c4, openwebtext, falcon-refinedweb, proof-pile-2, smollm-corpus + quality anchors
Sample Generations (Step 894,129)
⚠️ Model is at 33% of training. Outputs are expectedly noisy/incoherent at this stage.
| Prompt | Output (64 tokens, CPU inference) |
|---|---|
The capital of France is | Mz disc Topunion was resume abstra trop spacerrel Class WelshWangomas... |
def fibonacci(n): | Similarly form,meditortextttexpectTotal that. |
A polite greeting is | Fraction stringColor.Pathsrolledplay chips...frogrendvements numerals... |
2 + 2 = | 200 ofPMeurSUMzvements BandTotalcovering Delta quadrillixy Lagrange zoo... |
The theory of relativity states that | $$ European Numbers? QuantuplBlocksond. Total Cenidirectionalteness... |
Once upon a time | Mont?/no.redbit spectrometer Kidyunpless Cravements did Matlabitat... |
To make tea, first | quickly spodge quadrilexpect string-work... MatlabRemember that... |
The meaning of life is | Mz)Bvements and to handmadeuckedhel857Paths notchrelationships... |
Quality probe results (short-form, auto-infer):
- Promoted checkpoint (step 364,414): 3/4 probes passed ("Paris" ✅, "hello" ✅, code ✅, math ❌)
- Raw baseline (step 363,424): 0/4 probes passed
- Improvement trend confirmed; long-form coherence still developing
Full sample data: samples/agillm43_samples_step894129_20260706.json
Recovery Checkpoint Mirror
This repository mirrors recovery checkpoints from the fedC training run:
checkpoints/recovery_fedC/artifacts/full/— promoted full checkpointscheckpoints/recovery_fedC/artifacts/delta/— recovery backup deltaslatest_full.json,latest_delta.json— quality-gated uploader state
Quality Gate
- Perplexity/val CE: ELIMINATED (owner directive — model is undertrained, loss metrics caused 4+ regression-to-pin restarts)
- Promotion: Relative quality vs raw baseline (auto-infer probes)
- Resume: Newest delta unconditional (no rollback)
Links
- GitHub (public): Marxist-Leninist/AGILLM4.3
- Primary HF repo: OpenTransformer/AGILLM-4.3
- Training runtime: Single-file
agillm41.py
Latest generation sample (fedC recovery)
Captured 2026-07-06, ~30 min after cosine-LR decay went live on the fedC recovery run. Raw greedy generations from the auto-infer quality smoke, baseline pin (step 363424) vs current checkpoint (step 894634). Perplexity is intentionally not used as a quality signal; these are the actual sampled tokens.
| Prompt | Baseline 363424 (0/4) | Current 894634 (3/4) |
|---|---|---|
The capital of France is | , and "his bear toxiculeave | Paris. ✅ |
Python code to print hi: | (word-salad) | print "hello" ... def greet ✅ |
A polite greeting is | (word-salad) | Hello" ✅ |
2 + 2 = | and, - - t))), where | x. Python) print "hello" ❌ (still no arithmetic) |
Result: 3/4 probes now hit, up from 0/4 at the pin. The model is still early/undertrained
(fragmentary, punctuation-noisy, code/prose bleed) but is emitting on-target tokens — the first
qualitative jump after the constant-LR → cosine-decay fix. Checkpoint sha256 first64m:
d9669293ccd9ce3376dfa0e21439e070d3b9ee21bd773da8bd9b9c690365242b.
- Task
- Text Generation
- License
- other
- Created
- Aug 2, 2026
- Updated
- Aug 2, 2026