esp32-tinylm

esp32-tinylm

The 28.9M-parameter PLE TinyLM that runs on an ESP32-S3 microcontroller — as a transformers-loadable twin of the deployed model. These weights are dequantized bit-for-bit from the int4 artifact the chip executes (slvDev/esp32-ai, MIT), verified against the device's C runtime to ~1e-5. Not the training checkpoint: this is what the silicon actually computes.

• decoder-only, 6 layers, 4 heads, d_model 96, SwiGLU (66), RoPE, ctx 256
• Per-Layer Embeddings: 25M of the 28.9M parameters live in a per-layer lookup table (flash-resident on the device, ~450 B read per token)
• reads 32,768 token ids, writes 25,353 (tied head), ByteLevel BPE, English

Custom architecture (ple-tinylm): load it via the registration module in brainscope's ESP32 example, which serves it under a live logit lens / attention / steering UI:

pip install brainscope
python examples/esp32/serve.py

Sibling repo: esp32-tinylm-flash-unplugged — the same weights with the 25M flash-resident table zeroed. Credit: model and hardware story by slvDev; Per-Layer Embeddings are Google's Gemma 3n design; data is TinyStories.