Ling-3.0-tiny-EXL3-4bpw

Ling 3.0 Tiny — EXL3 4 bpw

EXL3 conversion of inclusionAI/Ling-3.0-tiny, revision e3a47d5b986e7141b6efd62597d598ebb392060d, for MLXL3 on Apple Silicon. The upstream model card declares MIT licensing and contains the original attribution and model details.

Conversion

• All 9,031 eligible projections use EXL3 K=4 and the MCG codebook, including 8,832 routed-expert projections and the vocabulary head.
• Embeddings, routers, convolutions, small-width projections and fused MLA kv_b_proj remain unquantized. Original FP32 tensors remain FP32; other unquantized tensors are exported in FP16.
• 4 bpw describes the quantized matrices, not the complete checkpoint.
• BF16 layerwise calibration: 2,048 real input rows, two independent 1,024-token sequences, with 128-token execution chunks and preserved cache.
• All experts receive common observed normalized MoE inputs. Each down projection is calibrated with its own expert's BF16 SwiGLU outputs, not another expert's activations. This is not routing-conditioned sampling.
• Metal LDLQ, 128-row buffers, 16-row feedback, sigma 0.025, no QAT.
• Original tokenizer, generation configuration and chat template retained. auto_map is removed from the exported config: MLXL3 uses native MLX-LM Bailing V3, not remote model Python.

Validation

source-perplexity.json and perplexity.json contain the BF16 and EXL3 WikiText-2 test results on the same 2,048 scored tokens, 256-token windows, and 128-token execution chunks: 21.0542 BF16 → 21.6694 EXL3 (+2.9%). All 9,031 EXL3 modules loaded; a short CLI generation answered 7 + 5 = 12. This small raw-text evaluation is not a general reasoning-quality benchmark. Quantization is lossy; no equivalence to a particular GGUF quality tier is claimed.

Run

Use an MLXL3 build containing the Ling adapter. Import the downloaded folder in the desktop model library, or register it in the CLI:

hf download 0xzknw/Ling-3.0-tiny-EXL3-4bpw --local-dir ./Ling-3.0-tiny-EXL3-4bpw
mlxl3 register ling-3.0-tiny-4bpw ./Ling-3.0-tiny-EXL3-4bpw
mlxl3 run ling-3.0-tiny-4bpw

The tokenizer's native thinking and tool-call template is preserved. MLXL3's Ling tool-call parser still requires users to enable the relevant MCP tools.