Qwen3.5-4B-MLX-4bit-text

Qwen3.5-4B · MLX 4-bit · text-only

Qwen3.5-4B for Apple Silicon, without the vision tower: 22 % smaller download, identical text output.

Why this exists

Many on-device apps use Qwen3.5-4B only for text: summarisation, extraction, chat. The original MLX conversion ships about 667 MB of vision weights that a text-only app downloads and never runs. This repo removes them and changes nothing else.

What changed

• Removed: the 297 vision_tower.* tensors from model.safetensors, plus the image/video processor configs (preprocessor_config.json, processor_config.json, video_preprocessor_config.json).
• Unchanged, byte for byte: every language-model tensor (same 4-bit quantization), config.json, the tokenizer and the chat template. config.json still declares the qwen3_5 architecture and its vision_config. MLX text loaders skip vision weights, so the repo loads as-is.

Verification

• mlx-swift-lm (LLMModelFactory): 23 real long-context extraction calls over two meeting transcripts, greedy decoding, thinking off. Every reply was byte-identical to the original repo's.
• mlx-lm: loads with mlx_lm.load and generates normally.

Quick start

from mlx_lm import load, generate

model, tokenizer = load("iktino/Qwen3.5-4B-MLX-4bit-text")
messages = [{"role": "user", "content": "Summarise this meeting in one sentence: ..."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False,
                                       enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

Swift (mlx-swift-lm): load it through LLMModelFactory with ModelConfiguration(id: "iktino/Qwen3.5-4B-MLX-4bit-text").

Limitations

• No image or video input. Use the original repo for multimodal work.
• Inherits every capability and limitation of Qwen3.5-4B. Read the original model card before production use.

License and attribution

Apache License 2.0, the same as the original model; LICENSE is copied unchanged from Qwen/Qwen3.5-4B. All credit for the model goes to the Qwen team. The MLX conversion is by mlx-community. The only change here is removing the vision weights.