SparkMuse-4B-MLX-4bit

SparkMuse-4B MLX 4bit

MLX 4-bit quantization of hcnote/SparkMuse-4B, a creative-writing / novel-generation / roleplay fine-tune of iFlytek Spark X2.5 (spark2_5 architecture, 1M context).

Quantization

• affine, 4-bit, group_size 64 (all 2D tensors including embedding)
• weights: 2.3 GB (BF16 source: 7.7 GB), resident ~2.5 GB
• speed on M2 Ultra (192 GB): ~129 tok/s decode (vs 60 tok/s for the BF16 original)
• tie_word_embeddings is handled by the engine (lm_head shares the quantized embedding)

Usage

Requires mlx-serve 26.9.5+ (native spark2_5 support). mlx-lm 0.31.x does not ship this architecture yet.

mlx-serve pull yachen4ever/SparkMuse-4B-MLX-4bit
mlx-serve serve yachen4ever/SparkMuse-4B-MLX-4bit --port 11234

OpenAI-compatible endpoint at http://127.0.0.1:11234/v1/chat/completions.

Notes

• The upstream repo is tagged nsfw / sensitive content by the author; the base model card describes a creative-writing oriented fine-tune. Use at your own discretion.
• The upstream modeling_spark.py / configuration_spark.py (transformers custom code) are not needed: the engine implements the architecture natively.