Downloads · 30 days
0
Inkling-imatrix-BF16-GGUF
satgeze/Inkling-imatrix-BF16-GGUF
Inkling-imatrix-BF16-GGUF is a machine learning model from satgeze. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
An importance matrix for TML's Inkling (975B MoE, 41B active), computed directly on the full BF16 weights. To our knowledge this is the first published imatrix for Inkling: the architecture is not yet merged into main…
Access
Public
Updated Jul 18, 2026
Repo size
1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.imatrix1 GB · 100%
At a glance
- License
- other
- Access
- Public
- Created
- Jul 18, 2026
- Updated
- Jul 18, 2026
- SHA
- e8911d3c
Base models
Model card
From the Hugging Face model README
Inkling imatrix (BF16-direct)
An importance matrix for TML's Inkling (975B MoE, 41B active), computed directly on the full BF16 weights. To our knowledge this is the first published imatrix for Inkling: the architecture is not yet merged into mainline llama.cpp, so standard quantization pipelines cannot produce one.
This repo contains ONLY the imatrix calibration file. There are no model weights here and no quantized builds are promised. If our VRAM-fit quantization experiments produce a build worth shipping, it will land in a separate repo aimed at memory-constrained rigs in the ~192 GB class (2x RTX PRO 6000, 2x DGX Spark and similar).
How it was made
| Source weights | BF16 GGUF (unsloth/inkling-GGUF, BF16/, 41 shards, ~1.8 TiB) |
| Collection | BF16-direct (not collected on a quantized model) |
| Calibration corpus | wikitext-2-raw train, full corpus |
| Chunks | 6,400 x 512 ctx = ~3.28M tokens |
| Build | llama.cpp PR #25731 (inkling architecture), CUDA build, CPU-streamed inference |
| Hardware | 2x RTX PRO 6000 Blackwell workstation, weights streamed from PCIe 5.0 NVMe |
| Wall time | ~10.6 hours |
BF16-direct collection matters at extreme quantization targets: importance statistics gathered from an already heavily quantized model are noticeably noisier, and this file is intended for 1-2 bpw work where that difference is largest.
Usage
llama-quantize --imatrix inkling-bf16.imatrix \
inkling-BF16-00001-of-00041.gguf inkling-IQ1_S.gguf IQ1_S
The file uses the GGUF container format written by current llama-imatrix. A build with the inkling architecture (PR #25731) is required for any end-to-end use with this model.
Notes
- Calibration text is English wikitext only. For heavily multilingual or code-dominant targets you may want to blend your own corpus.
- Made by satgeze. Feedback and results welcome in discussions.
- License
- other
- Created
- Jul 18, 2026
- Updated
- Jul 18, 2026