Qwen3.8-27B

Model Card for Qwen3.8-27B (GGUF, with vision)

GGUF builds of Qwen3.8-27B packaged to run entirely on device, with the vision tower, the tokenizer, the merge table, the chat template and the sampling card all embedded in the single weight file. Nothing calls out to a server at inference time and there are no sibling files to keep in sync.

This repository holds the quantized weights only, not training data or the original checkpoint.

File format

.ggxf is GGUF plus the tensors and metadata keys this project adds, so these files load in Gadeon and are not expected to load in other GGUF readers.

Files

Five points on the size/quality curve. Take ONE file, nothing beside it.

Every file carries the vision tower, the tokenizer, the chat template and the sampling card. The tower travels inside the GGUF rather than as a separate mmproj, and the sampling parameters travel with it, so a runtime never has to guess them or borrow another model's settings.

The iq4_nl column matters for third-party runtimes: every block type here is a standard ggml one inherited from the upstream quantization, but the two widest mixes include iq4_nl, which some runtimes decode and others do not.

The additions are the embedded vision tower, the drafter where the trunk lacked one, and the metadata. Those have only been exercised in the companion on-device runtime -- third-party tools have not been tested against these files, and a tool that does not expect a tower inside the weight file may ignore it or refuse it.

Provenance

Each language trunk is one of unsloth's quantizations of Qwen3.8-27B -- UD-IQ1_S, UD-IQ2_XXS, UD-IQ3_XXS, UD-IQ4_XS and UD-Q4_K_S -- used as published. What this build adds is the vision tower and the metadata: the v.* / mm.* tensors and clip.* configuration, the generation config with its sampling matrix and its stop-token list. The tower is carried at source precision (F16/F32) and is bit-identical to the tower in the origin checkpoint.

The UD-IQ3_XXS, UD-IQ4_XS and UD-Q4_K_S trunks ship unsloth's nextn layer already. UD-IQ2_XXS and UD-IQ1_S do not, so their drafter block was taken verbatim from unsloth's separate MTP/mtp-Qwen3.8-27B-Q4_0.gguf.

Model Details

Model Description

Qwen3.8-27B is a dense hybrid. Its layers alternate three Gated DeltaNet linear-attention blocks to one full attention block. The linear blocks carry a small fixed-size recurrent state instead of a growing key/value cache, so memory stays flat as context grows, while the periodic attention blocks preserve exact long-range recall.

These builds are multimodal. The vision tower is included, so image input works from any one of these files alone.

Every build carries a multi-token-prediction drafter. Speculative decode over it was measured token-identical to plain greedy on this repository's own runtime; a runtime that does not speculate decodes normally.

• Developed by: Qwen, Alibaba (base model); quantization by unsloth; packaging by leok7v
• Model type: Hybrid Gated DeltaNet + attention causal language model with a vision encoder
• Language(s): English and the languages of the base model
• License: Apache 2.0, inherited from the base model
• Quantized from model: Qwen/Qwen3.8-27B

Model Sources

• Repository (this build): https://huggingface.co/leok7v/Qwen3.8-27B
• Base model: Qwen/Qwen3.8-27B
• Trunk quantization: unsloth/Qwen3.8-27B-GGUF

Uses

Direct Use

On-device chat, summarization, drafting, question answering over supplied text, and description of supplied images, in offline or privacy-sensitive settings where prompts must not leave the device.

Out-of-Scope Use

Anything the base model's licence and card exclude. These builds add no alignment, no safety tuning and no filtering of their own; they are the upstream model in a different numeric format. Not a basis for legal, medical, financial or safety decisions, and not intended for high-stakes automated decisions without human review.

Bias, Risks, and Limitations

The base model's biases and failure modes carry over unchanged.

Quantization damage is real and it is not uniform across these files. The 1-bit build is the most heavily compressed in this family and is noticeably further from the upstream model than the wider ones: outputs will not match the upstream model token for token, the gap is widest where the model was already uncertain, and short factual questions can be answered incorrectly or declined even when the model "knows" the answer at higher precision. The 2-bit build is better but still visibly damaged. Long multi-step derivations are where errors accumulate fastest in every case.

Anyone depending on one of these builds should evaluate it on their own task rather than assume it behaves like the full-precision model, or like the other files here. If accuracy matters more than footprint, prefer a wider quant.

Recommendations

Treat generations as drafts to verify, not as fact. For anything where accuracy matters, and for arithmetic in particular, check the result.

How to Get Started with the Model

The tokenizer, the chat template and the sampling card are embedded, so a runtime reads all of them from the GGUF. The chat template documents the roles, the thinking block and the tool-call format.

The base model card recommends these sampling parameters, and the embedded sampling card carries them:

reasoning_effort is supported by the template; low and medium cost fewer tokens per turn than the default.

Generation stops on either <|im_end|> (248046) or <|endoftext|> (248044). Both ids are carried in the embedded generation config and in tokenizer.ggml.eos_token_ids; a runtime that reads only the scalar tokenizer.ggml.eos_token_id will see <|im_end|> alone.

Technical Specifications

Model Architecture and Objective

Weight format

GGUF v3. Every trunk is a MIXED-precision assignment rather than one block type throughout: the widths are chosen per tensor, so the most sensitive rows keep more bits than the bulk of the model. The name on the file is the dominant type, not the only one.

Dominant trunk types per file, by tensor count:

The Q4_K_S mix is worth a second look: despite the name, more of its trunk is IQ4_XS than Q4_K, and it is the file that carries the most iq4_nl.

In every file the norms, layer scalars and recurrent-state parameters stay F32 (360 tensors), and the 334 vision-tower tensors stay at source precision (222 F32 + 112 F16).

Citation

Please cite the upstream Qwen3.8-27B model. See the base model card at Qwen/Qwen3.8-27B for the canonical reference and license.

Model Card Contact

leok7v on Hugging Face.