Nexus-Erebus-50M

Nexus-Erebus-50M

Nexus-Erebus-50M is a compact ~54M-parameter language model trained from scratch by Ideoa Labs, combining strong commonsense reasoning with genuine integer arithmetic ability at tiny scale.

Model details

The tokenizer keeps digits atomic rather than merging them into BPE units, which preserves the positional structure that integer arithmetic depends on.

Benchmarks

Measured with lm-eval-harness, 0-shot, acc_norm, on the full test sets (no subsampling). ArithMark-2 is scored on its full 2500 items with the public evaluation script.

Against the sub-100M field

Published Open SLM Leaderboard values, same five tasks, same full-set protocol.

It leads the sub-100M class on ARC-easy and PIQA, and its ArithMark-2 score of 52.48 is the second highest in that class.

Training

Trained from scratch with a custom digit-aware 32k tokenizer, then refined on a curated mix of educational, science, commonsense and reading-comprehension data, plus a large synthetic integer arithmetic set covering addition, subtraction, multiplication, exact division, mixed multi-operator expressions and parenthesised expressions. No benchmark test items were used at any stage.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("MaliosDark/Nexus-Erebus-50M")
model = AutoModelForCausalLM.from_pretrained("MaliosDark/Nexus-Erebus-50M")

prompt = "16 + 4 * 3 ="
print(tok.decode(model.generate(**tok(prompt, return_tensors="pt"), max_new_tokens=6)[0]))

Example outputs

Real, unedited outputs from this checkpoint.

License

Apache-2.0.

Built by Ideoa Labs.