Downloads · 30 days
10
13% of all-time downloads
Inkling-GDN-Linearized
Zhongzhu/Inkling-GDN-Linearized
Inkling-GDN-Linearized is a machine learning model from Zhongzhu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Derivative of thinkingmachines/Inkling (Apache-2.0): the attention of layers [56, 58, 61, 62, 64, 65] (including full-attention layer 65) is replaced with GatedDeltaNet (64 heads × 128, gated, short-conv), trained by…
All-time downloads
76
Public
Parameters
954B
1.9 TB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors1.9 TB · 100%
Parameter types
How the weights are stored.
BF16954B · 100%
Base models
Model card
From the Hugging Face model README
Inkling-975B, 6 layers linearized to Gated DeltaNet (fused checkpoint)
Derivative of thinkingmachines/Inkling (Apache-2.0): the attention of layers [56, 58, 61, 62, 64, 65] (including full-attention layer 65) is replaced with GatedDeltaNet (64 heads × 128, gated, short-conv), trained by end-to-end KL distillation against the frozen base as teacher (1 800 steps, seq-128, 64×H100 EP). Layers were selected by zero-ablation importance ranking (least important first). All other weights are byte-identical to the base model.
| eval (Inkling-tokenized wikitext) | teacher (base) | this model | gap |
|---|---|---|---|
| PPL @seq-128 | 7.15 | 7.288 | +1.9% |
| PPL @seq-2048 (16× the training length) | 2.478 | 2.526 | +1.9% |
Format
Self-contained: the 109 original shards (native naming, unchanged) + model-gdn-linearized.safetensors
(keys model.layers.{L}.linear_attn.*) + patched model.safetensors.index.json.
config.json carries an xorl_linearization section describing the spliced layers.
Usage
Loadable with the xorl in-tree Inkling implementation
(xorl/models/transformers/inkling/, togethercomputer/xorl-internal PR #511):
build InklingConfig with linear_attn_layers=[56,58,61,62,64,65], linear_attn_type="gdn", kda_num_heads=64 and point weights_path at this repo — the bundled InklingCheckpointHandler
maps native keys (incl. gate/up de-interleave and expert EP-sharding) and passes the GDN keys through.
- Type
- inkling_mm_model
- License
- apache-2.0
- Created
- Jul 23, 2026
- Updated
- Jul 23, 2026