Aryabhata-2.0

Aryabhata 2

Aryabhata 2 is a reasoning-focused language model developed by PhysicsWallah for competitive STEM examinations (JEE, NEET). It is obtained by post-training GPT-OSS-20B via reinforcement learning on a curated curriculum of Physics, Chemistry, Mathematics, and General Reasoning questions — achieving strong accuracy at substantially lower inference cost than comparable models.

---

Model Summary

---

Performance

In-Distribution Benchmarks (Pass@1, 4-sample mean %)

Out-of-Distribution Benchmarks (Pass@1, 4-sample mean %)

Token Efficiency (Acc./1K tokens)

Aryabhata 2 achieves the best accuracy-per-token ratio of all evaluated models, using up to 64% fewer output tokens than GPT-OSS-20B.

---

Training Details

Data

The training corpus is derived from PhysicsWallah's internal question banks and processed through a multi-stage pipeline:

• Cleaning pipeline: HTML/image removal → LaTeX validation → LLM-based completeness check → domain filtering (~24% of data removed).
• Answer verification: Multi-pass sampling with GPT-OSS-120B as policy model and Qwen3-30B-A3B-Thinking as judge, covering 80% (1-sample), 8% (4-sample), and 4% (16-sample) of the dataset.

Methodology

Aryabhata 2 uses Group Relative Policy Optimization (GRPO) with LoRA adapters (rank 64, α=128), applied to attention projection and token embedding layers. Only 0.15% of parameters are trainable.

Reward function: R = R_accuracy × R_format, where accuracy uses a cascade of string, numeric, and symbolic matchers, and the format reward encourages well-structured, appropriately detailed responses.

Three-phase training:

LoRA Configuration

Read more about Aryabhata 2.0 in our Technical Report

---

Usage

System Prompt

SYSTEM_PROMPT = """
The user will provide a problem. Solve the problem. Explain step by step and put the final answer inside \\boxed{}

# Instructions
- The solution you provide in the final channel should be complete. The user should be able to follow your output step by step in order to get to the final answer.
- In case of Multiple Choice Questions, provide the option identifier as the final answer. (Example: \\boxed{B})
- In case multiple options are correct, provide the correct option identifiers, separated by semicolon (;). (Example: \\boxed{A;C})
- Put any units in \\text{} within in \\boxed{}. (Example: \\boxed{9.8\\ \\text{m/s}^2})
- The final answer should be in a single \\boxed{}
""".strip()

Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "PhysicsWallahAI/Aryabhata-2.0"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)


messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user",   "content": YOUR_QUERY},
]

input_ids = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

output = model.generate(
    input_ids,
    max_new_tokens=4096,
    temperature=1.0,
)

response = tokenizer.decode(
    output[0][input_ids.shape[-1]:],
)
print(response)

---

vLLM

from vllm import LLM, SamplingParams

model_id = "PhysicsWallahAI/Aryabhata-2.0"

llm = LLM(
    model=model_id,
    dtype="bfloat16",
    tensor_parallel_size=1,   # increase for multi-GPU
    max_model_len=16384,
)

sampling_params = SamplingParams(
    temperature=1.0,
    max_tokens=4096,
    skip_special_tokens=False,
)

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user",   "content": YOUR_QUERY},
]

outputs = llm.chat([messages], sampling_params)
print(outputs[0].outputs[0].text)

---

Intended Use

Primary use cases:

• Competitive exam preparation (JEE Main, JEE Advanced, NEET)
• STEM tutoring and student doubt resolution at scale
• Multi-step symbolic and numerical reasoning

---

Citation

@misc{aryabhata2,
  author       = {Rastogi, Ritvik and Singh, Vishal and Chaudhari, Tejas and Varma, Sandeep},
  title        = {Aryabhata 2},
  year         = {2025},
  publisher    = {PhysicsWallah},
  howpublished = {\url{https://huggingface.co/PhysicsWallahAI/Aryabhata-2.0}},
}

---

Contact

For questions, please contact [email protected] (PhysicsWallah).