Clyx_0.2-115.67M-BASE

Clyx 0.2 — 115.67M BASE

Causal language model trained from scratch on ~1.57B tokens.
Custom Transformer with RoPE, RMSNorm, SwiGLU. No external pretrained weights.

---

Architecture

---

Training

Data: Russian text, English text, Python and C/C++ code (~10 GB raw).

---

Inference

Base model — continues text, does not answer questions.

temperature        = 0.7
top_k              = 50
top_p              = 0.9
repetition_penalty = 1.15
max_new_tokens     = 1024

---

Files

---

Trained from scratch as part of the Clyx project.

How to Use

Base model — continues text, does not answer questions or follow instructions.

Option 1 — via ClyxBox

pip install clyxbox

from clyxbox import ClyxModel, ClyxTokenizer

tokenizer = ClyxTokenizer.from_pretrained("syntropic-clx/Clyx_0.2-115.67M-BASE")
model     = ClyxModel.from_pretrained("syntropic-clx/Clyx_0.2-115.67M-BASE")
model.eval()

prompt = "Once upon a time in a dark forest"
ids    = tokenizer.encode(prompt, return_tensors="pt")
out    = model.generate(ids, max_new_tokens=200, temperature=0.7, top_p=0.9)
print(tokenizer.decode(out[0], skip_special_tokens=True))

Option 2 — load weights manually

from safetensors.torch import load_file
state_dict = load_file("model.safetensors")

Special Tokens