Downloads · 30 days
0
jwl50-stanford/gdn-paper-120m-fixed-qk128-v64-fp32res-gpt2init-20k-r1
gdn-paper-120m-fixed-qk128-v64-fp32res-gpt2init-20k-r1 is a machine learning model from jwl50-stanford. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
This repository contains the final PyTorch Lightning checkpoint for the fixed-state Gated DeltaNet language-model experiment m06d10y26-gdn-paper-qk128-v64-fp32res-gpt2init-20k-r1.
Downloads · 30 days
0
Access
Public
Updated Jul 17, 2026
Repo size
1.7 GB
Likes
0
Public
Click a slice to open those files.
.ckpt1.7 GB · 100%
From the Hugging Face model README
This repository contains the final PyTorch Lightning checkpoint for the
fixed-state Gated DeltaNet language-model experiment
m06d10y26-gdn-paper-qk128-v64-fp32res-gpt2init-20k-r1.
The model has 12 layers, hidden size 768, and 12 heads. Each head uses a fixed Q/K feature dimension of 128 and a V dimension of 64. This is an ordinary fixed-state GDN: it does not use a growing-state mask or an expansion schedule.
last.ckpt: final checkpoint, byte-identical to the Perlmutter
step_20000.ckpt.train_config.yaml: exact training config preserved in the archived GDN
branch.eval_metrics.json: training/evaluation provenance, metrics, and the
checkpoint SHA-256 digest.m06d10y26-gdn-paper-qk128-v64-fp32res-gpt2init-20k-r1.54260616.9b40978e08d174bb24b1c675cdc1d2a655996ab4.HazyResearch/mlp-mixer commit
d2b24d685bc12f6e974d3ce68ffe2563a477cea1.768121212864 (expand_v=0.5)110.2112963416.2.3234939575 over 7,667,089,408 tokens.10.1007396226.10.1014004483.0.708458599321 over 1,156,539 selected
event tokens.This is a Lightning checkpoint for the custom mlp-mixer / Gated DeltaNet
training and evaluation stack, not a standalone Transformers-format model.
Use a compatible checkout of the preserved GDN branch and load last.ckpt
through that stack. The repository's existing checkpoint downloader convention
expects this filename.