tinyllm-cpu-char

tinyllm-cpu-char

A tiny CPU-trained character-level GPT-style language model created as an educational end-to-end LLM experiment.

This is not a general chatbot. It is a deliberately small model trained to demonstrate the full loop:

1. build a vocabulary
2. train a causal transformer
3. save a checkpoint
4. generate text from learned character patterns

Safety / provenance rule

This repo contains checkpoints created locally by us. We did not pull external model weights or external quote datasets into the training machine for the philosophy experiments. The philosophy corpus was extracted from existing local workspace philosophy book material, and a provenance manifest is included at data/philosophy_quotes_local_manifest.json.

Checkpoints

Original toy corpus checkpoint

• Checkpoint: checkpoints/tinyllm_overfit_3k.pt
• Architecture: tiny GPT-style causal transformer
• Tokenization: character-level
• Parameters: 106,688
• Training device: CPU
• Dataset: tiny included toy corpus, data/tiny_corpus.txt
• Training steps: 3,000
• Final loss: train 0.1903, val 3.7094

Local philosophy quote checkpoints

These were trained from local-only quote-style corpora derived from existing workspace philosophy files. They are intentionally tiny and educational.

Quick inference

pip install torch huggingface_hub
python hf_infer.py --repo-id Vishwas1/tinyllm-cpu-char \
  --filename checkpoints/tinyllm_philosophy_quotes_tiny50_3k.pt \
  --prompt "Wisdom" --tokens 300

For local inference after cloning:

python sample.py --ckpt checkpoints/tinyllm_philosophy_quotes_tiny50_3k.pt \
  --prompt "The soul" --tokens 300

Training examples

Original toy overfit:

python train.py --steps 3000 --eval-interval 500 --eval-iters 10 \
  --batch-size 16 --n-embd 64 --block-size 64 \
  --out checkpoints/tinyllm_overfit_3k.pt

Local philosophy tiny50 overfit-ish run:

python train.py --data data/philosophy_quotes_local_tiny50.txt \
  --steps 3000 --eval-interval 500 --eval-iters 10 \
  --batch-size 16 --n-embd 64 --block-size 64 --lr 0.001 \
  --out checkpoints/tinyllm_philosophy_quotes_tiny50_3k.pt

Limitations

• Character-level only.
• Generates memorized/stylized snippets, not reliable knowledge.
• Weird spelling and malformed words are expected.
• Intended for learning and experimentation.