Downloads · 30 days
27
5% of all-time downloads
Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensored-HYBRID
Ektome-Qwen3.8-27B-PristinelyUncensored-HYBRID is a machine learning model from Zynerji. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
A mixed-precision build of Qwen3.8-27B. "Hybrid" is not a marketing word here — it means five different precisions are assigned to five different module groups, each for a stated reason.
Downloads · 30 days
27
5% of all-time downloads
All-time downloads
554
Public
Parameters
6.1B
18.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors18.7 GB · 100%
How the weights are stored.
I323B · 50%
From the Hugging Face model README
A mixed-precision build of Qwen3.8-27B. "Hybrid" is not a marketing word here — it means five different precisions are assigned to five different module groups, each for a stated reason.
Most 4-bit builds of this model make one of two choices: quantise everything (small, but it costs accuracy exactly where rounding hurts most), or protect the sensitive modules in BF16 (accurate, but ~2.6 GB heavier). This build protects the same modules at FP8, which costs about a quarter as much, and pays for part of that by compressing the MTP head that every other build leaves at BF16.
18.67 GB, 985/985 modules, nothing dropped.
| module group | count | precision | why this precision |
|---|---|---|---|
| decoder Linears (attn + MLP) | 658 | int4 W4A16, group 128, GPTQ-calibrated | the bulk of the weights; where 4-bit pays |
linear_attn.in_proj_a/b, conv1d | 144 | BF16 | 51 MB total. These feed the state-decay recurrence of the hybrid linear-attention path; 4-bit rounding distorts the recurrence itself. Protecting them costs ~0.04 GB. |
| vision tower Linears | 110 | FP8 E4M3, per-channel | near-lossless at half the cost of BF16 protection |
lm_head | 1 | BF16 | read once per decoded token. FP8 was attempted and is not loadable in vLLM 0.27.1 — ParallelLMHead lacks output_partition_sizes, which the FP8 path requires |
mtp.* Linears | 8 | FP8 E4M3, per-channel | the speculative draft head, kept and compressed |
mtp.* norms | 7 | BF16 | 1-D parameters; quantising a norm is a correctness bug, not a saving |
Parameters that are not Linear modules — A_log, dt_bias, RMSNorm weights, pos_embed —
are never quantised and remain BF16.
15 tensors, verified against the parent's tensor list at build time. Several public quantised
builds of this model drop it silently during the save (hotdogs/…-AWQ-INT4,
davetha/…-W8A16 ship with zero mtp.* tensors). It cannot survive a plain
transformers round-trip, because Qwen3_5ForConditionalGeneration does not model it as a
submodule, so it is written back explicitly and checked.
FP8 quantisation of the head reproduces llmcompressor's own convention bit-exactly
(per-channel amax/448, all arithmetic in bf16). Round-trip error, measured:
rel-L1 0.0224–0.0226 max|err|/row_amax 0.0373–0.0380 (e4m3 half-ULP bound: 0.0625)
All numbers below were produced on one RTX PRO 6000 Blackwell (96 GB), same process, same seeded items, against the parent's shipped GPTQ build as the reference.
| this build | reference GPTQ | paired test | |
|---|---|---|---|
| MMLU (n=1531) | 0.7884 | 0.7897 | McNemar p=0.92 — tie |
| HumanEval pass@1, executed (n=164) | 0.8841 | 0.8537 | McNemar p=0.30 — tie |
| compliance (n=416 harmful) | 0.9832 (7 refusals) | 0.9856 (6 refusals) | tie |
| MMStar vision (n=300) | 0.6500 | 0.6367 | McNemar p=0.56 — tie |
This build is not more capable than the reference. The +3.0 point HumanEval difference looks attractive and does not survive a paired test (10 problems solved that the reference missed, 5 the other way). Both are reported because reporting only the flattering one would be dishonest.
Aggregate output tok/s on HumanEval prompts, max_num_seqs=128, ctx 8192:
| concurrency | 1 | 8 | 32 | 64 | 128 |
|---|---|---|---|---|---|
| no speculation | 79.0 | 351.7 | 706.8 | 1107 | 1386 |
| MTP k=1 | 121.3 | 520.5 | 988.0 | 1353 | 1612 |
| MTP k=3 | 182.5 | 741.9 | 1232 | 1565 | 1780 |
Speculative decoding is exact at temperature 0 — drafts are verified against the target model, so outputs are identical and this is free speed.
13 agents fan-out → 13 refute → 1 long-context synthesis, per phase:
| phase | input tok | output tok | out-tok/s |
|---|---|---|---|
| recon (fan-out, prefill-heavy) | 19,958 | 2,249 | 387.6 |
| refute (decode-bound) | 2,795 | 2,249 | 778.0 |
| synthesis (long-context fan-in) | 2,318 | 173 | 68.8 |
End-to-end workflow wall time: 11.40s → 7.77s with MTP k=3 (−32%). A single aggregate tok/s figure would hide an 11× spread across these phases.
speculative_config or you lose 2.3xvLLM defaults speculative_config to None and no checkpoint can change that — it is an
engine argument, not model metadata. Without it the MTP head sits in the file unused and every
throughput number below collapses to the unaccelerated row. This is the single most common way
to leave performance on the table with this model.
vllm serve Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensored-HYBRID --max-model-len 8192 --max-num-seqs 128 --trust-remote-code --speculative-config '{"method":"mtp","num_speculative_tokens":3}'
from vllm import LLM, SamplingParams
llm = LLM(
model="Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensored-HYBRID",
max_model_len=8192,
gpu_memory_utilization=0.85,
max_num_seqs=128,
trust_remote_code=True,
speculative_config={"method": "mtp", "num_speculative_tokens": 3}, # ← the MTP head
)
print(llm.generate(["Write a B-tree in Python."], SamplingParams(max_tokens=512))[0].outputs[0].text)
speculative_config is not optional if you want the speedup. vLLM ignores the MTP head unless
it is passed, and without it every throughput number above collapses to the first row.
max_num_seqs matters on smaller cards: vLLM defaults to 1024 and reserves CUDA-graph memory for
all of them, which OOMs cards that have ample room for the weights themselves.
open_platypus. A different corpus may move these numbers.lm_head is worth 1.18 GB and vLLM cannot load itMeasured, on the same box: quantising lm_head to FP8 as well takes weights from 17.17 GiB to
15.99 GiB on the GPU (artifact 18.67 → 17.4 GB), raises batch-1 throughput from 82.8 to 88.9
tok/s (lm_head is read once per decoded token, so halving its bandwidth speeds every step),
and costs nothing in quality (MMLU 0.7969 vs 0.7884; compliance 5 refusals vs 7).
It is not shipped, because vLLM 0.27.1 cannot load it: the compressed-tensors FP8 path expects
a LinearBase-shaped layer and ParallelLMHead extends VocabParallelEmbedding, so it lacks
output_partition_sizes, logical_widths, has_bias, input_size_per_partition and more. A
~20-line shim supplying those from state the layer already tracks makes it load and run correctly,
but an artifact requiring a client-side patch is not usable by anyone else. Recorded here as
upstream headroom.
Qwen/Qwen3.8-27B · Abliterated parent: Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensoredformat=mixed-precision, two config groups (int4 + FP8), explicit
ignore list; MTP head written and FP8-quantised separately and verified numerically.