Clyrai_Maximus-MoE

<p align="center"> <img src="maximus-hero.png" alt="Clyrai Maximus — 110 million parameters. Per token." width="100%"> </p> <br>

The rest of the model is silent.

Every language model you know lights up everything it has, for every word it says.

Maximus does not.

It is Clyrai's first Mixture of Experts. Two hundred and sixty-four million parameters on disk. One hundred and ten million fire for a token. Thirty-two specialists. Four speak. The others wait.

That is not a compression trick. That is the architecture.

---

Decode is the product.

Prefill is a parallel trick. Impressive. Irrelevant.

What you feel is the next token. Then the next. The GPU is not thinking. It is waiting on memory.

decode tok/s  =  bandwidth  /  (active parameters × 2 bytes)

So Maximus is not "a small model." It is a sparse machine.

Same law. Different pipe.

A dense 264M would drag every expert across that pipe. Maximus moves 2.4× less per word. Five hundred and four megabytes. The whole thing.

First token in 31 milliseconds. Then it streams.

---

Thirty-two experts. Four at a time.

No committee. A router.

Sigmoid. Top-4. One shared expert that always shows up. The first two layers stay dense — a quiet hallway before the hall of specialists.

It was trained that way from nothing. Not adapted. Not distilled from a giant. From scratch.

5.2 billion unique tokens. English. हिंदी. Italiano. Code. One hundred and twenty-eight thousand tokens of context.

Every expert lives. None of them died in training. That is rarer than it sounds.

---

Use it.

Maximus is a completion model. Give it a sentence that has already begun. Greedy decoding without a repetition penalty will loop — that is baked into generation_config.json so the defaults are the Clyrai defaults.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "saishshinde15/Clyrai_Maximus-MoE"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    trust_remote_code=True,
).cuda()

def complete(prompt: str) -> str:
    ids = tok(prompt, return_tensors="pt").to(model.device)
    out = model.generate(**ids, max_new_tokens=64)
    return tok.decode(out[0], skip_special_tokens=True)

print(complete("In a statement on Monday, the government"))
print(complete("भारत की राजधानी"))

trust_remote_code=True is required. This is not a dense Llama export. A Llama conversion would drop the experts.

MacBook (Apple Silicon)

No CUDA. Use MPS. On an 8 GB Air, quit other apps first — the weights are 504 MiB.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "saishshinde15/Clyrai_Maximus-MoE"
device = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
dtype = torch.float16 if device.type == "mps" else torch.float32

tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=dtype,
    trust_remote_code=True,
).to(device)

def complete(prompt: str) -> str:
    ids = tok(prompt, return_tensors="pt")
    ids = {k: v.to(device) for k, v in ids.items()}
    out = model.generate(**ids, max_new_tokens=64)
    return tok.decode(out[0], skip_special_tokens=True)

print(complete("In a statement on Monday, the government"))
print(complete("भारत की राजधानी"))

Or from this repo: python3 scripts/run_maximus_mac.py

---

One more thing.

This is the engine.

Not the assistant. Not the personality. Not the voice that says hello.

Clyrai built the sparse foundation first — because inference is the only moment the user is in the room. Chat comes after. The Mixture is already here.

---

The work is ours.

The weights are public. The title is not.

Study it. Cite it. Do not take it. Research and personal evaluation, with credit. Commerce, distillation, hosted product use, and training another model on Maximus require a grant from Clyrai.

All other rights reserved.

Clyrai Sovereign Research License 1.0

<p align="center"><br><em>Clyrai</em></p>

---

<details> <summary>The numbers, for the curious</summary> <br> </details>