Exeaon1-Dzo-4B

Exeaon-Dzo-4B

Compressed with E-PURE. Runs with the free epure-runtime, and stays compressed in memory — the dense weight is never assembled.

Quality

Measured against the base model on the same hardware, same harness version, same limit. Not copied from anyone's README.

Retention: 98.1% of base mean accuracy.

Note where the loss sits rather than reading only the mean: reasoning-heavy tasks degrade first, and a model chosen for retrieval or classification is affected differently from one chosen for reasoning.

Speed and footprint

Read this honestly: on GPU we are slower than dense fp16. A vendor tensor-core GEMM is extremely well tuned and a large GPU has bandwidth to spare, so dequantization costs more cycles than compression buys back. The win on GPU is footprint — whether the model fits, and what is left for the KV cache. On CPU, where bandwidth is the binding constraint, the fused kernel wins on both footprint and speed.

Measured on Tesla T4, torch 2.5.1+cu121, lm-eval 0.4.12, limit 300, kernel rust+triton.

Usage

pip install epure-runtime

from epure import load

model, tok = load("Exeaon/Exeaon-Dzo-4B")
ids = tok("Explain the memory wall.", return_tensors="pt").input_ids
print(tok.decode(model.generate(ids, max_new_tokens=128)[0]))

epure run Exeaon/Exeaon-Dzo-4B --prompt "Explain the memory wall."

No Rust toolchain needed: the fused CPU kernel ships compiled inside the wheel.

Fine-tuning without decompressing

Quantization indices stay frozen; the codebook and per-group scales train — about 1% of the weight values. The dense weight is never materialized, so memory tracks activations rather than parameter count.

from epure import load, make_trainable, snapshot_indices, verify_frozen

model, tok = load("Exeaon/Exeaon-Dzo-4B")
params, n = make_trainable(model, mode="both")
before = snapshot_indices(model)
...                              # an ordinary PyTorch loop over `params`
verify_frozen(model, before)     # raises if any index moved

Limitations

• Compression is lossy; the measured gap is in the table above.
• The KV cache is not compressed by default and can exceed the weights at long context.
• GPU decode is slower than dense fp16 (see above).
• Inherits every limitation, bias and knowledge cutoff of the base model.
• Not evaluated for safety-critical, medical, legal or financial use.

Licence and attribution

Derived from Qwen/Qwen3-4B, released under Apache-2.0, which permits redistribution of modified versions. The original LICENSE and NOTICE files are included in this repository unmodified, as it requires.

Ours: the compression method, calibration, packaging, runtime. Not ours: the pretrained knowledge, which comes from the base model authors. This model is not endorsed by or affiliated with them.

Citation

@misc{exeaon2026,
  title  = {Exeaon: compressed language models that run, and train, without
            decompressing},
  author = {Akpalu, Elliot Elikplim},
  year   = {2026},
  publisher = {Zenux Plimver Technologies LTD},
  url    = {https://huggingface.co/Exeaon}
}

---

_{Zenux Plimver Technologies LTD, Ghana}