nanochat-ar

nanochat-ar: Arabic LLM Proof Run on Aziz HPC

This repository contains a small Arabic nanochat proof-of-competence checkpoint trained from scratch on King Abdulaziz University Aziz HPC.

This is not a production Arabic LLM and does not claim state-of-the-art performance. The purpose is to document and release an end-to-end HPC training artifact: Arabic corpus processing, tokenizer training, base pretraining, evaluation, supervised fine-tuning, and checkpoint generation.

What is included

• Arabic tokenizer: tokenizer/tokenizer.pkl and tokenizer/token_bytes.pt
• Base checkpoint: base_checkpoints/ar-d12-eager/model_000050.pt
• SFT checkpoint: chatsft_checkpoints/ar-d12-eager/model_000002.pt
• Metadata JSON files for both checkpoints
• Training/evaluation reports in reports/
• Final HPC scripts in scripts/
• 200-example SFT sample in data_samples/

Optimizer states and the full raw corpus are not uploaded.

Training summary

How to load with nanochat

Clone karpathy/nanochat, install its environment, then use this repository as the NANOCHAT_BASE_DIR layout.

Commands:

export NANOCHAT_BASE_DIR=/path/to/this/repo
export NANOCHAT_DTYPE=float32
cd /path/to/nanochat
source .venv/bin/activate
python -m scripts.chat_cli --device-type cpu --source sft --model-tag ar-d12-eager --step 2 --prompt 'مرحبا، من أنت؟'

For GPU inference, use an A100 or similar GPU and the standard nanochat chat CLI/web server.

Limitations

• Very short proof training run: base model trained for only 50 steps.
• SFT is a compact generated instruction dataset, not a curated production instruction set.
• Chat quality is expected to be weak and repetitive.
• This release is best used as a reproducibility artifact and HPC workflow demonstration.

Citation / Acknowledgement

This project used King Abdulaziz University Aziz HPC resources. Built on karpathy/nanochat under the MIT license.