NeoHorse-1-9B-8bit

NeoHorse-1-9B MLX Quantized Variants

This repository contains four MLX-quantized variants of TokenRhythm/NeoHorse-1-9B, a 9B-parameter language model fine-tuned from Qwen3.5-9B for agentic post-training.

Model Details

Quantization Variants

Usage

mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("hermitdave/NeoHorse-1-9B-oQ4e")

messages = [{"role": "user", "content": "Write a Python function that returns the first n Fibonacci numbers."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
response = generate(model, tokenizer, prompt=prompt, max_tokens=512)

oMLX

omlx serve hermitdave/NeoHorse-1-9B-oQ4e --port 8000

CLI

python -m mlx_lm.chat --model hermitdave/NeoHorse-1-9B-oQ4e

Evaluation

NeoHorse-1-9B achieves a 69.04 macro average across ten benchmarks vs 65.60 for Qwen3.5-9B (+3.44). See the original model card for full evaluation details.

Conversion Notes

• oQ4e / oQ6e: Quantized using oMLX quantize_oq_streaming with imatrix calibration (128 samples, 512 seq length). These are mixed-precision formats — bit width varies per layer based on sensitivity measurement.
• 6-bit / 8-bit: Quantized using mlx_lm.convert with uniform affine quantization (group_size=64).
• MTP: The source checkpoint declares mtp_num_hidden_layers: 1 but contains no mtp.* weights (dropped during NeoHorse's repackaging from Qwen3.5-9B). These models do not include MTP heads.

Citation

@misc{neohorse2026,
  title        = {NeoHorse-1: Towards Recursive Self-Improvement via Agentic Post-Training with Routing Harness},
  author       = {NeoHorse Team},
  year         = {2026},
  howpublished = {arXiv preprint}
}

Acknowledgments

• TokenRhythm for the NeoHorse-1-9B model
• Qwen for the Qwen3.5-9B base model
• oMLX for the oQ quantization pipeline
• mlx-lm for uniform quantization
• Hermes Agent for conversion orchestration