Carnice-9b-MLX

Carnice-9b-MLX

4-bit MLX quantization of kai-os/Carnice-9b.

This conversion was produced using mlx_lm.convert for use with MLX on Apple Silicon.

Original Model

Carnice-9b by kai-os is a standalone merged model built on Qwen/Qwen3.5-9B, tuned specifically for the Hermes Agent harness. It was trained in two stages:

• Stage A — reasoning repair pass on high-signal reasoning data
• Stage B — Hermes-specific refresh pass built around harness-native traces and action structure

The model is optimized for terminal-heavy task execution, file editing, structured tool use, browser-assisted agent behavior, and multi-turn tool calling inside the Hermes runtime.

See the original model card for full training details and data sources.

Quantization Details

Model Architecture

Features

• Reasoning — supports <think> / </think> tags for chain-of-thought reasoning
• Tool calling — Hermes-style tool use with <tool_call> formatting
• Long context — 256K token context window
• Multimodal tokens — tokenizer includes vision and audio special tokens (inherited from Qwen3.5 base)

Benchmarks

Sampled evaluation (30 items per benchmark) on the 4-bit MLX quantized model:

Note: These are sampled results (30 items each), not full benchmark runs. They give a rough signal of quantized model quality but should not be compared directly to full-run scores.

Usage

With mlx-lm

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("jason-schulz/Carnice-9b-MLX")

prompt = "Explain the difference between linear and full attention."
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

response = generate(
    model,
    tokenizer,
    prompt=text,
    max_tokens=512,
)
print(response)

With MLX chat

mlx_lm.chat --model jason-schulz/Carnice-9b-MLX

Conversion

Produced with:

mlx_lm.convert \
    --hf-path kai-os/Carnice-9b \
    --mlx-path Carnice-9b-MLX \
    -q \
    --q-bits 4 \
    --q-group-size 64 \
    --q-mode affine

Conversion Note

The original model's config.json uses model_type: "qwen_3_5_text", which mlx_lm.convert does not recognize. The conversion will fail with an unsupported model type error. To work around this, manually change model_type to "qwen3_5" in the source model's config.json before running the conversion. The resulting model loads and runs correctly under MLX.

Credits

• Original model: kai-os/Carnice-9b by kai-os
• Base model: Qwen/Qwen3.5-9B by Qwen
• MLX conversion: This repository