nanochat-hebrew-d20

nanochat-hebrew-d20

This repo hosts multiple versions of a Hebrew nanochat model. Each version is self-contained: its base/ (and sft/ where present) weights are paired with the matching tokenizer/ in the same subfolder. Tokenizers are not interchangeable across versions.

Versions

v1-mixed-75he-25en/

First Hebrew nanochat. Pretrained on a 75% Hebrew / 25% English mixture of OSCAR-23.01 (he), Hebrew Wikipedia, CC-100 (he), Knesset corpus, and FineWeb-Edu English. Tokenizer includes English. No SFT, no nikkud stripping.

v2-fw2-hebrew-only/

Pure Hebrew nanochat. Pretrained on HuggingFaceFW/fineweb-2 config heb_Hebr only, with nikkud (U+0591-U+05C7) stripped before tokenization. Tokenizer is Hebrew-only. Followed by SFT on HeQ, ParaShoot, Dolly-HE, OpenHermes-2.5-Hebrew, plus hand-templated identity conversations.

Loading

Each version uses nanochat's native format. Point nanochat.checkpoint_manager. load_model_from_dir at the version subfolder after downloading:

from huggingface_hub import snapshot_download
from nanochat.checkpoint_manager import load_model_from_dir

local = snapshot_download("ShFishman/nanochat-hebrew-d20",
                         allow_patterns=["v2-fw2-hebrew-only/*"])
# point tokenizer loader at v2's tokenizer dir before calling load_model_from_dir