sprapp-smollm2-360m-ternary

sprapp-smollm2-360m-ternary

Ternary (≈1.58-bit) quantization-aware-trained build of SmolLM2-360M, packaged in the KNM1 format for the Sprapp in-browser engine (Rust → WebAssembly). Runs fully on-device / offline — no server, no API.

What this is

• Base: SmolLM2-360M (base, not instruct) → use plain text continuation, not a chat template.
• Quantization: linear projection weights ternary {-1, 0, +1} with a per-tensor absmean scale (2-bit packed, 4 weights/byte). Embeddings/LM-head kept f16; norms full precision. Trained with QAT (straight-through estimator) on ~2.8B tokens so the network adapts to the ternary weights rather than being post-hoc rounded.
• Format: model_smol.knm — KNM1 v3 (header: dim 960, 32 layers, 15/5 GQA heads, head_dim 64, hidden 2560, vocab 49152, non-interleaved RoPE).
• Engine: decoded by a hand-written WASM kernel (gemv_ternary) — the 2-bit weights are dotted against int8-quantized activations, so the whole 360M model generates in a browser tab.

Quality (eyecheck, greedy)

Once upon a time, there was a little  →  girl named Emily who lived in the woods. She had always
   been fascinated by her mother's old wooden cabin and felt that it would be nice to have one too...
The capital of France is              →  Paris, and the country's largest city is Paris. The French
   language has a rich history dating back to ... Latin ...
Water boils at a temperature of       →  102 degrees Fahrenheit, which is the boiling point for water...

Fluent and largely on-topic. Factuality is at the level you'd expect from a 360M base model under aggressive quantization (e.g. it says water boils at 102°F — fluent, not exact).

Files

Usage

Load model_smol.knm with the Sprapp WASM engine and generate by plain continuation at low temperature (≈0.2). It is a base LM — feeding an instruct/chat template makes it emit end-of-turn immediately.

Limitations

• Base model: no instruction following, no chat.
• Aggressive ternary quantization trades some factual precision for size/speed.
• 360M params at ternary is still ~220 MB on disk (the f16 embedding table for the 49k vocab is the bulk); serve from object storage, not a 25 MB-capped static host.

Part of the Sprapp project — offline on-device tiny/quantized LMs in the browser.