Downloads · 30 days
27
44% of all-time downloads
gemma4-12b-base-text-backbone
kennethgrace/gemma4-12b-base-text-backbone
gemma4-12b-base-text-backbone is a machine learning model from kennethgrace. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as gemma.
The language backbone of google/gemma-4-12B, extracted as a standalone text model.
All-time downloads
62
Public
Parameters
11.9B
23.8 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors23.8 GB · 100%
At a glance
- License
- gemma
- Model type
- gemma4_unified_text
- Access
- Public
- Created
- Jul 15, 2026
- Updated
- Jul 15, 2026
- SHA
- be2b0771
Base models
Model card
From the Hugging Face model README
gemma4-12b-base-text-backbone
The language backbone of google/gemma-4-12B, extracted as a standalone text model.
The vision and audio towers (model.visual.*, embed_vision, embed_audio,
vision_embedder, …) and their config are removed; the text decoder
(model.language_model.*, with lm_head tied to the embeddings) is repackaged
as a Gemma4UnifiedForCausalLM. Because Gemma-4 is early-fusion, a text-only
forward through the full multimodal model uses only these weights — so this
backbone is the substrate you get whether you strip before or after a text-only
fine-tune, minus the multimodal plumbing.
Faithfulness
Verified against the original model's text-only forward:
max |Δ logit| = 0.00e+00 with 100% next-token argmax agreement across
prose, interior-monologue (with CIS structural tokens), code, and ASCII-grid
(ARC) probes. Same decoder tensors, same text_config — no RoPE reconstruction,
no weight permutation.
License
This is a Gemma derivative and is distributed under the Gemma Terms of Use, not Apache-2.0.
Produced by tools/extract_gemma4_text_backbone.py (Kenneth Grace & Kira).
- Type
- gemma4_unified_text
- License
- gemma
- Created
- Jul 15, 2026
- Updated
- Jul 15, 2026