Qwen3.5-9B

Model Card for Qwen3.5-9B (4-bit GGUF, with vision)

A 4-bit GGUF build of Qwen3.5-9B packaged to run entirely on device, with the vision tower, the tokenizer, the merge table, the chat template and the sampling card all embedded in the single weight file. Nothing calls out to a server at inference time and there are no sibling files to keep in sync.

This repository holds the quantized weights only, not training data or the original checkpoint.

File format

.ggxf is GGUF plus the tensors and metadata keys this project adds, so these files load in Gadeon and are not expected to load in other GGUF readers.

Files

One file, nothing beside it. The vision tower travels inside the GGUF rather than as a separate mmproj, and the sampling parameters travel with it, so a runtime never has to guess them or borrow another model's settings.

Every block type here is a standard ggml one, inherited from the upstream quantization. The additions are the embedded vision tower and the metadata, and those have only been exercised in the companion on-device runtime -- third-party tools have not been tested against this file, and a tool that does not expect a tower inside the weight file may ignore it or refuse it.

Provenance

The language trunk and the multi-token-prediction drafter are unsloth's UD-Q4_K_XL quantization with the MTP block of Qwen3.5-9B, used as published. What this build adds is the vision tower and the metadata: the v.* / mm.* tensors and clip.* configuration, and the generation config with its sampling matrix and its stop-token list. The tower is carried at source precision (F16/F32).

Model Details

Model Description

Qwen3.5-9B is a dense hybrid. Its layers alternate three Gated DeltaNet linear-attention blocks to one full attention block. The linear blocks carry a small fixed-size recurrent state instead of a growing key/value cache, so memory stays flat as context grows, while the periodic attention blocks preserve exact long-range recall.

This build is multimodal. The vision tower is included, so image input works from this file alone.

This build carries the multi-token-prediction layer (blk.32, the nextn block), so a runtime that speculates can draft from it; one that does not decodes normally.

• Developed by: Qwen, Alibaba (base model); quantization by unsloth; packaging by leok7v
• Model type: Hybrid Gated DeltaNet + attention causal language model with a vision encoder
• Language(s): English and the languages of the base model
• License: Apache 2.0, inherited from the base model
• Quantized from model: Qwen/Qwen3.5-9B

Model Sources

• Repository (this build): https://huggingface.co/leok7v/Qwen3.5-9B
• Base model: Qwen/Qwen3.5-9B
• Trunk quantization: unsloth/Qwen3.5-9B-MTP-GGUF

Uses

Direct Use

On-device chat, summarization, drafting, question answering over supplied text, and description of supplied images, in offline or privacy-sensitive settings where prompts must not leave the device.

Out-of-Scope Use

Anything the base model's licence and card exclude. This build adds no alignment, no safety tuning and no filtering of its own; it is the upstream model in a different numeric format. Not a basis for legal, medical, financial or safety decisions, and not intended for high-stakes automated decisions without human review.

Bias, Risks, and Limitations

The base model's biases and failure modes carry over unchanged.

Four bits is a mild compression compared with the 1-bit and 2-bit builds in this family, but it is still lossy: outputs will not match the upstream model token for token, and the gap is widest where the model was already uncertain. Evaluate on your own task rather than assuming it behaves like the full-precision model.

Recommendations

Treat generations as drafts to verify, not as fact. For anything where accuracy matters, and for arithmetic in particular, check the result.

How to Get Started with the Model

The tokenizer, the chat template and the sampling card are embedded, so a runtime reads all of them from the GGUF. The chat template documents the roles, the thinking block and the tool-call format.

Generation stops on either <|im_end|> (248046) or <|endoftext|> (248044). Both ids are carried in the embedded generation config and in tokenizer.ggml.eos_token_ids; a runtime that reads only the scalar tokenizer.ggml.eos_token_id will see <|im_end|> alone.

Technical Specifications

Model Architecture and Objective

Weight format

6.57 GiB, 776 tensors (334 of them the vision tower at F16/F32), GGUF v3. The trunk is a MIXED-precision assignment rather than one block type throughout: the widths are chosen per tensor, so the most sensitive rows keep more bits than the bulk of the model.

Citation

Please cite the upstream Qwen3.5-9B model. See the base model card at Qwen/Qwen3.5-9B for the canonical reference and license.

Model Card Contact

leok7v on Hugging Face.