Orca-2-4B-hight

Orca-4B-Agentic

Faster • Cheaper • Stronger for agentic & coding tasks

Optimized successor of the original Rhea-4B-Coding / Athenea line.

What changed for better price/performance

→ Same 4B base intelligence, noticeably faster and cheaper to run, better real-world agentic behavior because it is no longer forced into three full generations.

Why this is better for agentic tasks

• Standard <think>...</think> for chain-of-thought (optional, model decides when useful)
• Clean, reliable tool-calling format
• No artificial multi-pass overhead that multiplies latency and cost
• Strong coding + reasoning focus retained
• System prompt encourages precise, secure, efficient solutions

Technical specs

• Architecture: Qwen3ForCausalLM (4B)
• Context: 32 768 tokens
• Special tokens: <|im_start|>, <|im_end|>, <think>, </think>, tool tags
• Recommended quant: Q4_K_M / AWQ for best speed/quality

Recommended settings

{
  "temperature": 0.4,
  "top_p": 0.9,
  "top_k": 30,
  "repetition_penalty": 1.05,
  "max_new_tokens": 8192
}

Quickstart

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "./Orca-2-4B-high"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype="auto",
    device_map="auto"
)

messages = [
    {"role": "user", "content": "Write a secure Python function that validates JWT tokens and handles expiration gracefully."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=2048)
print(tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=False))

Deployment tips (max cheapness)

• vLLM / SGLang: --max-model-len 32768
• llama.cpp: Q4_K_M or Q5_K_M
• Keep context ≤ 16k–24k in production for optimal speed/VRAM

Credits

• Base: Qwen3-4B + Athenea / Rhea coding lineage
• Optimized for agentic use, single-pass thinking, lower cost

Apache 2.0