Mellum2-12B-A2.5B-Instruct-AWQ-INT4

<div align="left"> <a href="https://cyan.kiwi" target="_blank"><img src="https://storage.googleapis.com/cyankiwi-assets/model-card/model-card-banners-cyankiwi.png" width="800" alt="cyankiwi, the LLM inference agent for open models"></a> </div> <div align="left"> <table style="border-collapse:collapse; border:none;"> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Website</b></td> <td align="left" style="border:none; padding:4px 0;"><a href="https://cyan.kiwi/" target="_blank">cyan.kiwi</a></td> </tr> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Models</b></td> <td align="left" style="border:none; padding:4px 0;"><a href="https://huggingface.co/cyankiwi" target="_blank">Hugging Face</a> · <a href="https://www.modelscope.ai/organization/cyankiwi" target="_blank">ModelScope</a></td> </tr> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Connect</b></td> <td align="left" style="border:none; padding:4px 0;"><a href="https://x.com/cyankiwi_ai" target="_blank">X</a> · <a href="https://www.reddit.com/r/cyankiwi/" target="_blank">Reddit</a> · <a href="mailto:[email protected]">Email</a></td> </tr> </table> </div> <br> <div align="left"> <img src="https://storage.googleapis.com/cyankiwi-assets/model-card/model-card-banners-awq.png" width="800" alt="cyankiwi Activation-aware Weight Quantization"> </div> <div align="left"> <table style="border-collapse:collapse; border:none;"> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Version</b></td> <td align="left" style="border:none; padding:4px 0;">cyankiwi 26.05.01</td> </tr> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Calibration</b></td> <td align="left" style="border:none; padding:4px 0;"><a href="https://huggingface.co/datasets/cyankiwi/calibration" target="_blank">STEM and Agentic</a></td> </tr> <tr style="border:none;"> <td align="left" style="border:none; padding:4px 24px 4px 0;"><b>Model Size</b></td> <td align="left" style="border:none; padding:4px 0;">10.46 GB</td> </tr> </table> </div> <br> <p align="left"> <a href="https://cyan.kiwi/workspace?prompt=Deploy%20cyankiwi/Mellum2-12B-A2.5B-Instruct-AWQ-INT4" target="_blank"><img src="https://storage.googleapis.com/cyankiwi-assets/model-card/model-card-button-deploy.png" width="333" alt="Deploy on cyan.kiwi private endpoints"></a> </p>

---

<img alt="Mellum" src="mellum-logo-dark.svg" width="320">

Mellum2 Instruct

[!Note] Use this model when you want direct, low-latency answers without an explicit chain of thought — interactive chat, code assistance, tool use, and instruction following. If you need explicit reasoning before the answer (complex debugging, planning, multi-step agentic flows), use Thinking instead.

Mellum2 Instruct Highlights

Mellum2 Instruct is a post-trained assistant model trained by JetBrains.

The model uses a Mixture-of-Experts architecture with 64 experts and activates 8 experts per token. It uses a combination of sliding-window and full attention layers, with a context length of 131,072 tokens.

It is produced from Mellum2-12B-A2.5B-Base by supervised fine-tuning followed by reinforcement learning with verifiable rewards (RLVR) on math, executable coding, tool use, instruction following, reasoning, and knowledge tasks. Mellum2 Instruct answers directly, without an externalized chain of thought.

Mellum2 Model Family

This repository contains one checkpoint from the Mellum2 family.

Model Overview

Mellum2 Instruct has the following features:

• Number of Layers: 28
• Hidden Size: 2304
• Intermediate Size: 7168
• MoE Intermediate Size: 896
• Number of Experts: 64
• Number of Activated Experts: 8
• Number of Attention Heads (GQA): 32 for Q and 4 for KV
• Context Length: 131,072
• Sliding Window: 1,024
• Vocabulary Size: 98,304
• Precision: bfloat16

Serving with vLLM

# Without tool calling
vllm serve JetBrains/Mellum2-12B-A2.5B-Instruct --max-model-len 131072

# With tool calling
vllm serve JetBrains/Mellum2-12B-A2.5B-Instruct \
  --max-model-len 131072 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes

Quickstart

Text-Only Input

from openai import OpenAI
# Configured by environment variables
client = OpenAI()

messages = [
    {"role": "user", "content": "Write a Python function to reverse a string."},
]

chat_response = client.chat.completions.create(
    model="JetBrains/Mellum2-12B-A2.5B-Instruct",
    messages=messages,
    max_tokens=81920,
    temperature=0.6,
    top_p=0.95,
    extra_body={
        "top_k": 20,
    },
)
print("Chat response:", chat_response)

Evaluation

Post-training evaluation for the instruct (no-thinking) variants. All values are percentages; higher is better except HarmBench, where lower is better. All values self-reported by JetBrains.

Notes:

• EvalPlus is the mean of HumanEval+ and MBPP+.
• AIME is the mean of AIME 2025 and AIME 2026 (30 questions each).
• BFCL v4 is the macro-average of five subtasks: v1, v2, v3, web search, memory.
• JetBrains pairwise is win rate against Qwen2.5-7B-Instruct on an internal benchmark.
• — indicates the model lacks native tool calling.

For more details, see the Mellum2 Technical Report.

License

Released under the Apache 2.0 license.