Downloads · 30 days
0
disk-routed-ssd-chat-0.5b
jaivial/disk-routed-ssd-chat-0.5b
disk-routed-ssd-chat-0.5b is a machine learning model from jaivial. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Experimental disk-routed recurrent language model designed to run inference on CPU + SSD only, with host RSS well under 1 GiB and no GPU. Model capacity lives in an int8 external memory table paged from SSD via ODIREC…
Access
Public
Updated Jul 13, 2026
Repo size
522 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.i8500 MB · 95%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jul 13, 2026
- Updated
- Jul 13, 2026
- SHA
- 914f776c
Model card
From the Hugging Face model README
Disk-Routed SSD Chat 0.5B
Experimental disk-routed recurrent language model designed to run inference on CPU + SSD only, with host RSS well under 1 GiB and no GPU. Model capacity lives in an int8 external memory table paged from SSD via O_DIRECT (2 rows/token); a small selective-recurrent controller stays resident in RAM.
Facts
| Item | Value |
|---|---|
| Total weights | 505,437,952 |
| External (SSD) weights | 499,968,000 |
| Controller weights | 5,469,952 |
| Tokenizer | 32,000 BPE (FineWeb-Edu) |
| Base training tokens | 500,000,016 (FineWeb-Edu) |
| Alignment | 20k UltraChat SFT + 500-step DPO |
Measured inference speed (CPU only, no GPU)
Hardware: AMD Ryzen 9 5950X, NVMe SSD, int8 external table paged with O_DIRECT. Runtime is the C decoder direct_chat.c. "BPE tok/s" = generated 32k-BPE tokens per second.
| Mode | Output speed | I/O per token | Process RSS | GPU |
|---|---|---|---|---|
Cold, SSD-persisted recurrent state (disk-state) | 774 BPE tok/s | 16 KiB | 23.0 MB | none |
| In-RAM recurrent state (state kept in RAM, weights on SSD) | 2,916 BPE tok/s | 8 KiB | 23.3 MB | none |
| Chat eval worst-case sustained | 1,172 BPE tok/s | 8-16 KiB | <25 MB | none |
Notes:
- Every configuration is >50x the 15 tok/s target and holds RSS far below the 1 GiB budget.
- Only 2 int8 rows (2 x 4 KiB pages) are read per token regardless of table size; optional 2 more pages persist recurrent state to SSD.
- Speed is bounded by per-token SSD page reads plus small controller compute, not by total model size.
Honest limitations
This is a systems proof, not a quality model. Chat evaluation semantic pass rate was ~0.40 with visible repetition and BPE spacing markers. 500M tokens and a ~5.5M-active controller are insufficient for reliable chat. It demonstrates that ~0.5B-weight capacity can be served from SSD at hundreds-to-thousands of tok/s under 25 MB RAM.
Files
memory.i8- int8 external table, page-aligned (499,970,048 bytes)controller.f32- controller weights,CDR1headerscale.txt- int8 dequant scalemanifest.json- config and byte countsfineweb-tokenizer.json- tokenizer
Runtime: a C O_DIRECT decoder (direct_chat.c) in the project repo.
- License
- apache-2.0
- Languages
- en
- Created
- Jul 13, 2026
- Updated
- Jul 13, 2026