Downloads · 30 days
12
30% of all-time downloads
vicharak-micro-llama
weights-and-wires/vicharak-micro-llama
vicharak-micro-llama is a text generation model from weights-and-wires. Use it when you need the model to write or continue text. It is set up for flax. The card lists the license as apache-2.0.
Ultra-small Llama-style language model trained on TinyStories for bare-metal inference on the Vicharak Shrike-Lite (RP2040 + Renesas FPGA).
All-time downloads
40
Public
Repo size
1.9 MB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.bin853 KB · 44%
At a glance
- Task
- Text Generation
- Library
- flax
- License
- apache-2.0
- Model type
- micro-llama
- Access
- Public
- Created
- Aug 3, 2026
- Updated
- Aug 3, 2026
- SHA
- 10ff8e56
Try a prompt
Datasets
Model card
From the Hugging Face model README
Micro-Llama for Vicharak Shrike-Lite
Ultra-small Llama-style language model trained on TinyStories for bare-metal inference on the Vicharak Shrike-Lite (RP2040 + Renesas FPGA).
Weights are sized to live in QSPI Flash via XIP; activations + KV cache fit in the RP2040's 264KB SRAM.
Model details
| Field | Value |
|---|---|
| Parameters | 213,312 |
d_model | 64 |
| MLP hidden | 128 |
| Layers | 2 |
| Heads | 2 |
| Context | 128 |
| Vocab (BPE) | 1024 |
| Architecture | RoPE · RMSNorm · SwiGLU |
| Framework | JAX / Flax / Optax |
| Train device | gpu |
| Train steps | 5,000 |
| Dataset | roneneldan/TinyStories |
Files
| File | Description |
|---|---|
micro_llama.bin | llama2.c-style float32 weight blob (Phase-3 firmware) |
micro_llama.json | Export metadata / layout notes |
latest_params.npz | Flax parameter tree (NumPy object array) |
latest_meta.json | Train + model config used for this checkpoint |
tinystories_bpe_1024.model | SentencePiece BPE model |
tinystories_bpe_1024.vocab | SentencePiece vocabulary |
Training summary
- Optimizer: AdamW (β₁=0.9, β₂=0.95), cosine decay after warmup
- Batch size: 32
- Peak LR: 0.0003
- Weight decay: 0.1
- Grad clip: 1.0
- Final eval loss: 3.2735
- Final eval perplexity: 26.4023
- Card generated: 2026-08-03 13:44 UTC
Intended use
Research / education for edge LLM inference on MCU+FPGA boards. Not intended for general chat or production NLP.
How to use (Python)
# from this repo
uv run python phase-2/generate.py --prompt "Once upon a time"
Or load the SentencePiece tokenizer + Flax params from this repo and run MicroLlama in phase-2/model.py.
How to use (firmware)
- Flash
micro_llama.bininto the Phase-3 C inference engine (Pico SDK / UF2). - Stream tokens over USB serial from the Shrike-Lite.
Limitations
- Tiny capacity: expects simple children’s-story English (TinyStories distribution).
- Context limited to 128 tokens.
- FP32 export by default; quantize (INT8) before aggressive on-device acceleration.
Citation
@misc{micro-llama-shrike,
title = {Micro-Llama for Vicharak Shrike-Lite},
author = {weights-and-wires},
year = {2026},
url = {https://huggingface.co/weights-and-wires/vicharak-micro-llama}
}
Acknowledgements
- roneneldan/TinyStories
- llama2.c export conventions
- Vicharak Shrike-Lite hardware platform
- Task
- Text Generation
- Library
- flax
- Type
- micro-llama
- License
- apache-2.0
- Languages
- en
- Created
- Aug 3, 2026
- Updated
- Aug 3, 2026