ZaryaOrthrus-0.6B

Zarya Orthrus 0.6B

This is a checkpoint of the Orthrus hybrid-architecture model based on Qwen3-0.6B. It was trained using the code available in the Zarya-Orthrus repository and the data available in the ZaryaOrthrusDataset-1.7B dataset.

Tokens Per Forward

The table below reports the speculative efficiency of the model, measured as the number of tokens generated per autoregressive forward pass.

Getting Started

The following code demonstrates how to generate a response using the model:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
 

model_id = "ai-forever/ZaryaOrthrus-0.6B"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    dtype=torch.bfloat16, device_map="cuda",
    attn_implementation="flash_attention_2",
    trust_remote_code=True,
).eval()
tokenizer = AutoTokenizer.from_pretrained(model_id)
 
prompt = "Explain me what is Special Theory of Relativity, who invented it and what is its essence."

messages = [{"role": "system", "content": ""}, {"role": "user", "content": prompt}]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True, enable_thinking=False).input_ids

output_ids = model.generate(
    input_ids=input_ids.to(model.device), 
    max_new_tokens=512,
    use_diffusion_mode=True, 
    streamer=TextStreamer(tokenizer, skip_prompt=True) # enable streaming generation
)