LongCat-2.0-2bit

pipenetwork/LongCat-2.0-2bit

2-bit (2.501 bits/weight) MLX quantization of meituan-longcat/LongCat-2.0, a 1.6T-parameter / ~48B-active MoE (MLA attention + LongCat sparse-attention indexer + identity experts + n-gram embeddings). Converted from the FP8 source with mlx-lm. Router classifiers are kept at 8-bit (mixed precision); MTP layers are dropped.

Size: ~477 GB. This exceeds a 512 GB unified-memory ceiling in practice — intended for larger-memory or sharded/multi-node MLX inference, not a single 512 GB machine.

Requires mlx-lm PR #1464

LongCat-2.0 (model_type: longcat2) support is not yet in a released mlx-lm. Install from the PR branch:

pip install git+https://github.com/ml-explore/mlx-lm.git@refs/pull/1464/head

Use

from mlx_lm import load, generate

model, tokenizer = load("pipenetwork/LongCat-2.0-2bit")
messages = [{"role": "user", "content": "Who is Albert Einstein?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512, verbose=True))

For large builds, use sharded/distributed generation (mlx.launch + sharded_generate.py).