qwen38-mtp-head-fc-bf16-4bit

junafinity/qwen38-mtp-head-fc-bf16-4bit

qwen38-mtp-head-fc-bf16-4bit is a text generation model from junafinity. Use it when you need the model to write or continue text. It is set up for mlx. The card lists the license as apache-2.0.

Downloads · 30 days

19% of all-time downloads

All-time downloads

329

Public

Parameters

111M

314 MB on disk

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors314 MB · 100%

Parameter types

How the weights are stored.

BF1664.1M · 58%

Try a prompt

Base models

EigenLabs/Qwen3.8-27B-MTP-bf16

Model card

From the Hugging Face model README

Built by VeriSafe.Live

Open-weights models, built, fine-tuned, quantized and served.

</div>

What VeriSafe.Live does

Offering	What you get
🎯 Custom fine-tuning	Any open-weights model adapted to your data, domain and tone.
🧪 Training from scratch	Purpose-built models designed and trained for your use case.
⚡ Quantization (MLX / GGUF)	Lean builds that run fast on Apple Silicon and llama.cpp.
🌐 Hosted API access	Instant access to served models, with no wait time.
🔓 Uncensored models	Refusal-removed builds for research, red teaming and evaluation.

🚀 Now live at VeriSafe.Live: uncensored DeepSeek V4.1 Flash 1017, GLM 5.3 Flash and GLM 5.3. No wait time, instant access, crypto accepted.

Model highlights

Optional native-MTP drafting head for Qwen3.8-27B on MLX.
Keeps fc in bf16 and quantizes the other seven linears to 4-bit/g64.
An experiment; acceptance on hard prose is unmeasured.

Qwen3.8-27B MTP head — mixed precision (bf16 `fc`, 4-bit/g64 elsewhere)

A drafting head for Qwen 3.8 27B native-MTP speculative decode on MLX / Apple Silicon. This is an optional experiment, not a speed product. Acceptance on hard, varied prose is unmeasured. Do not quote tok/s for this file. Derived from the organizer-pinned EigenLabs/Qwen3.8-27B-MTP-bf16 @ 26a328e070875b0314d652a039b6b59902690f03.

Pairing

Hub collection: https://huggingface.co/collections/junafinity/qwen-38-27b-uncensored-apple-silicon-6a896c726b52be3a0b63400e

This is a drafting head only, not a standalone model. Pair it with a Qwen3.8-27B MLX backbone (for example Qwen-3.8-27B-Uncensored-8-Bit-MLX) in a loader that accepts a scales-keyed mixed-precision head.

backbone: junafinity/Qwen-3.8-27B-Uncensored-8-Bit-MLX
draft head: junafinity/qwen38-mtp-head-fc-bf16-4bit

Exact loader flags: [PLACEHOLDER] (depends on your mlx / speculative-decode entrypoint). Acceptance-rate benefit on hard, varied prose is unmeasured; see the caveat below.

What is different

Existing 4-bit heads for this model quantize all eight head linears. This one keeps fc in bf16 and quantizes only the other seven (q/k/v/o_proj, gate/up/down_proj) to MLX affine 4-bit, group size 64 — the geometry that matches the 4-bit backbone, so a stock scales-keyed loader converts exactly the modules that carry a .scales sibling and leaves fc a dense Linear with no code changes.

Why `fc`

Qwen's own official FP8 release, Qwen/Qwen3.8-27B-FP8, lists mtp.fc in modules_to_not_convert — it declines to quantize that matrix while quantizing the head's other large linears. fc is the projection that fuses the embedding stream with the backbone hidden state (concat[embed, hidden] at 10240 → 5120), so representation error there propagates into every downstream head computation at every draft position. This artifact tests whether respecting that exclusion improves draft acceptance enough to pay for the extra bandwidth.

Cost/benefit

fc in bf16 is 104.9 MB versus 29.5 MB at 4-bit — about +75 MB of weight traffic per draft step (total head read ≈ 314 MB vs ≈ 239 MB). Break-even needs only a small acceptance gain at mid/deep draft positions, since an accepted draft token amortizes a full ~15.6 GB target verify pass on this dense 27B model.

29 tensors: fc.weight (bf16, [5120, 10240]), seven bf16 norm vectors, and weight/scales/biases triples for the seven quantized linears. Single model.safetensors (314,300,360 bytes) plus config.json and an index, carried unchanged from the pinned head so the head loader accepts the tree.

Caveat on evaluation

Short local decode windows over a model's own greedy self-continuation saturate draft acceptance (measured: effective draft length 5.4, accept rate 1.000, identical for this head and a uniformly 4-bit one). Such a setup can measure this artifact's cost but not its benefit; evaluation needs harder, more varied prose where acceptance is genuinely below 1.

💬 Need a custom model?

If your company or research institute would like us to fine-tune any open-weights model, or train one from scratch, we'd be happy to help.

Drop us a line at [email protected]