Sovythos-APEP-550M

<p align="center"> <img src="https://huggingface.co/my0919175/Sovythos-APEP-550M/resolve/main/apep.png" width="640" alt="SOVYTHOS APEP"> </p> <h1 align="center">🐍 SOVYTHOS — APEP</h1> <h3 align="center">The Chaos-Serpent Foundation Model of the Sovythos AI Project</h3> <p align="center">

</p>

---

🚧 This repository is a placeholder — training is currently in progress

This repo has been created ahead of time to reserve the model name and document what's coming. No weights, tokenizer, or config files are uploaded yet. This README exists purely to explain the situation to anyone who finds this repo early.

Nothing here is runnable yet. AutoModelForCausalLM.from_pretrained(...) will fail against this repo until the files below are pushed. Watch/follow the repo, or check back later, for the real release.

---

🐍 What is APEP?

SOVYTHOS — APEP is a ~550M-parameter, Llama-compatible decoder-only Transformer, trained entirely from scratch (random init — no Llama/GPT/Qwen weights anywhere in its lineage) on a single-stage mixture of:

• Egyptian Arabic + Modern Standard Arabic (authentic dialect sources, not machine-translated)
• English, including high-signal educational web text (FineWeb-Edu, Cosmopedia-v2)
• Mathematics (raw math text + step-by-step solved problems)
• Code (quality-filtered Python + instruction-formatted tasks)
• Instruction-formatted examples mixed in from the very first training step (not a separate SFT stage)

It's the current flagship of the Sovythos AI Project, trained end-to-end — data pipeline, custom tokenizer, architecture, and training loop — on a single consumer-class GPU.

Named after Apep (Apophis), the serpent of chaos in Egyptian mythology who rises nightly against the sun and is beaten back only through relentless, disciplined effort — a fitting name for a model built to stay coherent despite limited compute and a demanding multilingual mixture.

---

📋 What's happening right now

---

📦 What will be uploaded once training finishes

• model.safetensors — the trained weights
• config.json — full architecture config (dim, layers, heads, vocab, etc.)
• tokenizer.json — the custom-trained tokenizer
• modeling_sovythos.py + configuration_sovythos.py — so the model loads directly via AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True) with no external code needed
• sovythos-apep-550m.f16.gguf — for llama.cpp / llama-cpp-python inference
• train_log.csv — the full training curve (train/val loss over steps), for transparency
• A complete, detailed README replacing this placeholder — specs, architecture breakdown, tokenizer design, data sources, training recipe, usage examples, GGUF instructions, limitations, and roadmap

---

⏳ In the meantime

• This repo will not be usable for inference until the files above land
• Model name and repo ID (my0919175/Sovythos-APEP-550M) are final and won't change
• Training progress is tracked internally via train_log.csv and will be shared once uploaded
• Follow this repository to be notified when the first real checkpoint goes up

---

👨‍💻 Creator

Mahmoud Yasser — Founder, Sovythos AI Project

---

📄 License

Apache License 2.0 (applies once weights are released)

---

<p align="center">

🐍 <b>The serpent is still coiling. Check back soon.</b>

</p>