Ling-3.0-flash-MLX-4bit

<p align="center"> <img src="https://mdn.alipayobjects.com/huamei_qa8qxu/afts/img/A*4QxcQrBlTiAAAAAAQXAAAAgAemJ7AQ/original" width="100" alt="Ling logo"> </p> <h1 align="center">Ling 3.0 Flash — MLX 4-bit</h1> <p align="center"> A native Apple-silicon conversion of <a href="https://huggingface.co/inclusionAI/Ling-3.0-flash">inclusionAI/Ling-3.0-flash</a>, quantized with the stock MLX-LM converter and packaged for MLX-LM and oMLX. </p> <p align="center"> <a href="https://huggingface.co/inclusionAI/Ling-3.0-flash">Original model</a> · <a href="https://modelscope.cn/organization/inclusionAI">InclusionAI on ModelScope</a> · <a href="https://openrouter.ai/inclusionai/ling-3.0-flash:free">OpenRouter</a> </p>

About this conversion

This repository contains a 4-bit affine MLX conversion of Ling 3.0 Flash. Ling is a 124B-parameter hybrid-linear mixture-of-experts model with approximately 5.1B active parameters per token. The conversion preserves the upstream tokenizer and chat template and includes an MLX architecture adapter for the model's KDA, MLA, and sparse-MoE layers.

[!IMPORTANT] This model includes the custom architecture file bailing_hybrid.py. Review the file and enable Trust Remote Code before loading it. In oMLX, enable Trust Remote Code in the model settings. With MLX-LM, pass --trust-remote-code.

Apple-silicon validation

The converted weights were loaded strictly with both stock MLX-LM and the MLX runtime bundled with oMLX. End-to-end generation was also tested through the oMLX OpenAI-compatible API on an Apple-silicon Mac Studio.

One short local smoke test produced:

These figures confirm that the model runs; they are not a controlled benchmark. Prompt length, sampler settings, macOS memory pressure, thermal state, and MLX/oMLX versions can materially change performance.

Quick start with MLX-LM

Install recent MLX-LM and Hugging Face tooling:

python -m pip install -U mlx-lm huggingface_hub

Run a chat prompt directly from the Hub:

mlx_lm.generate \
  --model Vontra/Ling-3.0-flash-MLX-4bit \
  --trust-remote-code \
  --prompt "Explain why hybrid linear attention is useful." \
  --max-tokens 512 \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20

Thinking mode is enabled by the upstream chat template by default. It can be disabled with:

mlx_lm.generate \
  --model Vontra/Ling-3.0-flash-MLX-4bit \
  --trust-remote-code \
  --chat-template-config '{"enable_thinking": false}' \
  --prompt "Write a short hello-world program in Swift."

To download the repository first:

hf download Vontra/Ling-3.0-flash-MLX-4bit \
  --local-dir ~/.omlx/models/Vontra/Ling-3.0-flash-MLX-4bit

Using it with oMLX

1. Place the model at ~/.omlx/models/Vontra/Ling-3.0-flash-MLX-4bit.
2. Refresh the oMLX model registry.
3. Open the model settings and enable Trust Remote Code.
4. Load Ling-3.0-flash-MLX-4bit and use the normal chat or OpenAI-compatible endpoint.

Example request:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OMLX_API_KEY" \
  -d '{
    "model": "Ling-3.0-flash-MLX-4bit",
    "messages": [{"role": "user", "content": "Say hello from MLX."}],
    "temperature": 0.6,
    "top_p": 0.95,
    "max_tokens": 128
  }'

For long prompts, begin with a conservative context limit and increase it while watching memory pressure. The advertised 256K context is a model capability, not a promise that every host can prefill that context within its available unified memory.

Architecture

Ling 3.0 Flash is a native hybrid-linear MoE model. It alternates Kimi Delta Attention (KDA) and gated Multi-head Latent Attention (MLA) in a 5:1 ratio and uses highly sparse routed experts.

The included MLX adapter uses MLX-LM's native primitives for delta attention, absorbed MLA projections, RoPE, and quantized SwitchGLU experts. The auxiliary MTP training head is intentionally excluded from ordinary causal generation; this release does not claim MTP/DSpark speculative decoding support.

Upstream model highlights

InclusionAI describes Ling 3.0 Flash as a next-generation hybrid reasoning model designed for efficient long-context inference and agentic workloads. The upstream release reports strong results across software-engineering agents, tool use, deep research, general knowledge, mathematical reasoning, instruction following, and long-context understanding.

The upstream defaults are:

thinking: enabled
temperature: 0.6
top_p: 0.95
top_k: 20

For benchmark methodology, scores, limitations, and framework-specific SGLang/vLLM deployment instructions, see the original InclusionAI model card.

Conversion notes

• Source weights: BF16 upstream checkpoint.
• Conversion command: stock mlx_lm.convert with --quantize --q-bits 4 --q-group-size 64 --q-mode affine.
• The upstream chat_template.jinja is included unchanged.
• All source tensors required for base causal generation were consumed by a strict load before conversion.
• The 42 base transformer layers are retained. The separate depth-5 MTP training/speculation stack is not part of this runtime adapter.

Four-bit weight quantization materially reduces memory use but may reduce output quality relative to BF16 or the 8-bit conversion.

This is a community conversion, not an official InclusionAI release. Because the architecture adapter is new, validate quality and numerical behavior on your own representative workload before production use.

License and attribution

The upstream model is released under the MIT License. This conversion preserves that license and is derived from inclusionAI/Ling-3.0-flash.

All model design, training, and benchmark credit belongs to InclusionAI and the original contributors. The MLX conversion and compatibility adapter are provided by Vontra.