Ornith-1.5-9B-MixedInt4-AutoRound

Ornith-1.5-9B-MixedInt4-AutoRound

A mixed-precision AutoRound quantized version of ornith-ai/Ornith-1.5-9B, optimized to reduce memory requirements while preserving the capabilities and quality of the original model.

Base Model

Base model: ornith-ai/Ornith-1.5-9B

This model is a quantized version of the original Ornith-1.5-9B checkpoint. It is not a fine-tune, merge, or distillation.

Ornith-1.5-9B is a dense multimodal reasoning model based on the Qwen3.5 architecture.

Quantization

Quantization was performed using Intel AutoRound with a custom mixed-precision quantization configuration.

The quantization recipe was designed to balance:

• Model quality
• Memory efficiency
• Inference performance
• vLLM compatibility

Some model components are intentionally retained at higher precision where appropriate.

The calibration process was primarily optimized for instruction-following performance[no-think], focusing on maintaining high-quality responses for user instructions and conversational interactions after quantization.

The detailed mixed-precision allocation strategy is not included in this model card.

Evaluation

Evaluation was performed using AutoRound's evaluation interface with LM Evaluation Harness.

The following results compare the original Ornith-1.5-9B model against Ornith-1.5-9B-MixedInt4-AutoRound.

MMLU Category Breakdown

GSM8K

Recovery Rate represents benchmark performance relative to the original Ornith-1.5-9B checkpoint. A recovery rate above 100% indicates that the quantized model scored higher than the original model in that particular evaluation. Benchmark preservation does not imply identical behavior for every prompt, multimodal workload, long-context workload, agentic workload, or generation setting.

Usage

This checkpoint is intended for inference engines with AutoRound quantization support, including compatible versions of vLLM.

Ornith-1.5-9B is a reasoning model and supports tool calling. Recent versions of the serving runtime are recommended.

vLLM

vllm serve Pilcothink/Ornith-1.5-9B-MixedInt4-AutoRound \
  --served-model-name Ornith-1.5-9B-MixedInt4-AutoRound \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --enable-prefix-caching \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90

Recommended Sampling Parameters

For general tasks:

temperature=1.0
top_p=0.95
top_k=20
min_p=0.0
presence_penalty=1.5
repetition_penalty=1.0

For precise coding tasks:

temperature=0.6
top_p=0.95
top_k=20
min_p=0.0
presence_penalty=0.0
repetition_penalty=1.0

Long-Context Serving

The native context length of Ornith-1.5-9B is 262,144 tokens.

For workloads requiring a context window beyond the native limit, Ornith recommends extending the effective context using YaRN RoPE scaling.

Example for approximately 1M tokens:

VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 \
vllm serve Pilcothink/Ornith-1.5-9B-MixedInt4-AutoRound \
  --served-model-name Ornith-1.5-9B-MixedInt4-AutoRound \
  --host 0.0.0.0 \
  --port 8000 \
  --trust-remote-code \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --enable-prefix-caching \
  --gpu-memory-utilization 0.90 \
  --hf-overrides '{"rope_scaling":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144}}' \
  --max-model-len 1000000

Static YaRN scaling is applied to all requests and may slightly affect quality on shorter contexts. It should therefore only be enabled when workloads genuinely require context lengths beyond the native 262,144-token window.

Notes

• This is a quantized derivative of ornith-ai/Ornith-1.5-9B.
• The model uses a custom mixed-precision INT4 quantization recipe.
• The vision components are preserved at their original precision.
• The model retains the original Ornith reasoning and tool-calling format.
• The native context length is 262,144 tokens.
• Quantization may introduce small behavioral differences compared with the original checkpoint.
• Long-context, multimodal, reasoning, and agentic workloads may respond differently from the original model even when benchmark recovery is high.

Acknowledgements

• Base model: Ornith / DeepReinforce Team
• Underlying architecture: Qwen3.5
• Quantization framework: Intel AutoRound

Please refer to the original ornith-ai/Ornith-1.5-9B model card for licensing, intended usage, limitations, recommended sampling parameters, and other information applicable to the base model.