Apriel-1.5-15b-Thinker-2bit-MLX

Apriel-1.5-15B-Thinker — MLX 2-bit (Apple Silicon)

Format: MLX (Mac, Apple Silicon)
Quantization: 2-bit (ultra-compact)
Base: ServiceNow-AI/Apriel-1.5-15B-Thinker
Architecture: Pixtral-style LLaVA (vision encoder → 2-layer projector → decoder)

This repository provides a 2-bit MLX build of Apriel-1.5-15B-Thinker for tight-memory Apple-Silicon devices. It prioritizes small footprint and fast load over absolute accuracy. If quality is your primary concern, prefer the 6-bit MLX variant.

---

🔎 What is Apriel-1.5-15B-Thinker?

Apriel-1.5-15B-Thinker is an open multimodal reasoning model that scales a Pixtral-style VLM with depth upscaling, two-stage multimodal continual pretraining (CPT), and high-quality SFT with explicit reasoning traces (math, coding, science, tool-use). The training recipe focuses on mid-training (no RLHF/RM), delivering strong image-grounded reasoning at modest compute.

This card documents the 2-bit MLX conversion. Expect higher compression and noticeable quality drop vs FP/Int or 6-bit, especially on fine-grained text in images, dense charts, or long-chain reasoning.

---

📦 What’s in this repo

• config.json (MLX config mapped for Pixtral-style VLM)
• mlx_model*.safetensors (2-bit quantized shards)
• tokenizer.json, tokenizer_config.json
• processor_config.json / image_processor.json
• model_index.json and metadata

---

✅ Intended uses

• On-device image understanding where memory is constrained (light captioning, object/layout descriptions)
• Quick triage of screenshots, UI mocks, simple charts, forms with broad structure
• Educational demos of VLMs on Mac w/ minimal RAM budget

⚠️ Limitations

• 2-bit is very lossy. Expect degradation on:
  • OCR-heavy tasks, small fonts, dense tables
  • Multi-step math/coding with visual grounding
  • Long context or many images
• May hallucinate or miss small details. Human review is required for critical use.

---

🖥️ Apple-Silicon guidance

• Works: M1/M2 (8–16 GB) for short prompts + single image; recommended: M3/M4 for smoother throughput.
• Use GPU:

  --device mps