Argonne-2.5-ctx13568-instruct

Argonne-2.5-ctx13568-instruct

Argonne-2.5-ctx13568-instruct starts from the long-context checkpoint PursuitOfDataScience/Argonne-2.5-ctx13568 and is tuned in two stages: long-context SFT followed by DPO.

Model architecture

Finetuning pipeline

Stage 1 was supervised fine-tuning on HuggingFaceH4/ultrachat_200k using the local train_sft export at /project/rcc/youzhi/data/HuggingFaceH4_ultrachat_200k/train_sft. That run used max_seq_length=13568, batch_size=1, grad_accum=4, lr=2e-5, num_epochs=1, and warmup_steps=100, and produced the intermediate checkpoint final_model_sft_long.

Stage 2 was DPO on KatoHF/chatbot_arena_binarized using the local export at /project/rcc/youzhi/data/KatoHF_chatbot_arena_binarized with the chat_refine_strict recipe. That run used max_seq_length=13568, batch_size=4, grad_accum=8, lr=5e-6, num_epochs=5, warmup_steps=10, beta=0.2, score_mode=avg, label_smoothing=0.0, and chosen_sft_weight=0.1. The published checkpoint corresponds to /project/rcc/youzhi/llm.c/checkpoints/final_model_dpo_long.

The released weights are stored in bfloat16 and published as 5 safetensor shards.

Training data

• Base checkpoint: PursuitOfDataScience/Argonne-2.5-ctx13568
• SFT data: HuggingFaceH4/ultrachat_200k (train_sft)
• DPO data: KatoHF/chatbot_arena_binarized (chat_refine_strict)

Tokenizer

This model uses the Qwen3 tokenizer family via the Qwen2Tokenizer compatibility class.

Source code

The release was built from the GitHub main branch codebase: https://github.com/PursuitOfDataScience/ArgonneAI/tree/main

Key scripts:

• sft.py
• dpo.py
• model.py

Recommended inference config

These settings mirror the sampled generation path used in dpo.py for quality checks.

Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "PursuitOfDataScience/Argonne-2.5-ctx13568-instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
)

device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device)
model.eval()

messages = [
    {"role": "user", "content": "Explain what a black hole is in a way a 10-year-old would understand."}
]
prompt_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
)
input_ids = torch.tensor([prompt_ids], dtype=torch.long, device=device)

seed = 444
torch.manual_seed(seed)
if device.startswith("cuda"):
    torch.cuda.manual_seed_all(seed)

output_ids = model.generate(
    input_ids,
    max_length=min(model.config.max_position_embeddings, input_ids.shape[1] + 200),
    temperature=0.8,
    top_p=0.9,
    do_sample=True,
    repetition_penalty=1.3,
    no_repeat_ngram_size=4,
)
gen_ids = output_ids[0, input_ids.shape[1]:].tolist()
eos_id = tokenizer.eos_token_id
if eos_id in gen_ids:
    gen_ids = gen_ids[: gen_ids.index(eos_id)]

reply = tokenizer.decode(gen_ids, skip_special_tokens=True).strip()
print(reply)

Usage notes

• Load with trust_remote_code=True.
• Use the chat template via tokenizer.apply_chat_template(..., add_generation_prompt=True) for instruct prompting.
• The custom generate method uses max_length, so the example trims the continuation at tokenizer.eos_token_id after generation.
• Weights are published as 5 bf16 safetensor shards.
• The instruct checkpoint inherits the tokenizer and chat template from the long-context base model.

Citation

@misc{argonne25ctx13568instruct,
  author = {PursuitOfDataScience},
  title = {Argonne-2.5-ctx13568-instruct},
  year = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/PursuitOfDataScience/Argonne-2.5-ctx13568-instruct}
}