Downloads · 30 days
0
gemma-4-12B-it-assistant-coreai
warshanks/gemma-4-12B-it-assistant-coreai
gemma-4-12B-it-assistant-coreai is a machine learning model from warshanks. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
INT4 Core AI (.aimodel) conversion of google/gemma-4-12B-it-assistant, the multi-token-prediction draft model for Gemma 4 12B-it. Companion to warshanks/gemma-4-12B-it-coreai — used for speculative decoding in Wyvern…
Access
Public
Updated Jun 12, 2026
Repo size
1.6 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.mlirb804 MB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Jun 12, 2026
- Updated
- Jun 12, 2026
- SHA
- 5c9bb246
Base models
Model card
From the Hugging Face model README
Gemma 4 12B-it Assistant — Core AI (MTP draft model)
INT4 Core AI (.aimodel) conversion of google/gemma-4-12B-it-assistant, the multi-token-prediction draft model for Gemma 4 12B-it. Companion to warshanks/gemma-4-12B-it-coreai — used for speculative decoding in Wyvern Chat's on-device Core AI provider (macOS 27+, Apple silicon).
Contents
Single inference function draft:
| name | shape | dtype | |
|---|---|---|---|
| input | input_ids | [1, 1] | int32 |
| input | backbone_hidden | [1, 1, 3840] | bf16 |
| input | position_ids | [1, S] | int32 |
| state | k_cache / v_cache | [48, 1, 8, ctx, 256] | bf16 |
| output | next_token | [1, 1] | int32 |
| output | hidden | [1, 1, 3840] | bf16 |
The draft cross-attends the main model's KV cache (layers 46/47) — pass
the same Metal buffers used for the main bundle, zero-copy. backbone_hidden
is the main model's post-final-norm hidden state (the hidden output of the
main bundle's main/prefill_multimodal functions). The bundle embeds an
INT4 copy of the main model's embedding table, so each draft step needs only
the previous token id, not its embedding.
Drafted greedily (in-graph argmax over [0, 255999) — special/multimodal tokens are never proposed). ~3 ms/step on an M4 Max vs ~21 ms for the 12B.
Conversion
Exported with Apple's coreai-torch / coreai-models toolchain (INT4 block-32 weight-only, symmetric with clipping). Numerics verified against the HF reference implementation (logits max-abs-err 3e-4 at S=1500).
Modifications from the original weights: INT4 quantization; the unused
centroid masked-embedder path is dropped (use_ordered_embeddings: false);
the main model's embedding table is bundled in.
License
Apache 2.0, same as the base model. Original copyright Google DeepMind. See LICENSE and NOTICE.
- License
- apache-2.0
- Created
- Jun 12, 2026
- Updated
- Jun 12, 2026