Thomson-1.0-Small-MLX-4bit

Thomson-1.0-Small — MLX 4-bit Quantization

MLX format conversion of thomsonreuters/Thomson-1.0-Small, a 35B parameter Mixture-of-Experts VLM with hybrid linear attention.

Base architecture: Qwen3.5-35B-A3B (hybrid attention MoE, 256 experts, 8 active per token) Vision tower: Qwen3.5 Vision Encoder, kept at fp16 (not quantized)

Model Details

Quickstart

pip install -U mlx-vlm

python3 -m mlx_vlm.generate \
  --model hermitdave/Thomson-1.0-Small-MLX-4bit \
  --prompt "Summarize the key points of this document." \
  --max-tokens 512 --temp 1.0 --top-p 0.95

For deterministic outputs, use temp=0. For complex reasoning tasks, increase max-tokens.

Conversion Details

• Tool: mlx-vlm v0.6.17 convert (lazy mmap mode)
• Quantization: Uniform affine, RTN, no mixed-predicate
• Vision tower: Preserved at fp16 via skip_multimodal_module predicate
• Dtype: bfloat16
• Hardware: Apple M3 Max 64 GB
• Conversion script: convert_thomson_4b_6b.py

Attribution

Upstream model: thomsonreuters/Thomson-1.0-Small by Thomson Reuters.

Thomson-1.0 stem: tri-fair-lab/Snowdon1.1-Small by Tair Lab.

Conversion: Hermes Agent (Nous Research) using mlx-vlm v0.6.17 on Apple Silicon.

Technical report: "Thomson: Continual Learning of Frontier Models for SovereignAI" (arXiv:2608.27147).

Other Formats

Also available: 6-bit variant

License

Same as upstream: Qwen3.6 Community License. See upstream repo for full terms.

Disclaimer

Quantized models may exhibit slightly different behavior compared to the original BF16 checkpoint. Please validate for your specific use case.