Argonne-3.0-think

Argonne 3.0-think

Argonne 3.0-think is a 2.88B-parameter reasoning model from the Argonne 3.x family. It supports an explicit chain-of-thought "think" mode (<think> … </think>) and answers directly in no-think mode. It is built entirely from Argonne 3.0-base through a multi-stage pipeline that ends in training-free weight-soups — one to repair the pretraining base, one to reconcile reasoning with general chat, and further recovery soups after each round of self-improvement.

On a 40-item internal 4-quadrant probe ({math, general} × {no-think, with-CoT}) it scores 33/40, with strong arithmetic in both modes while remaining a loop-free general chat model. (This is a small hand-checked probe — for held-out benchmark numbers and how to get the most out of the model, see Getting more accuracy: test-time compute below.)

📝 Companion blog post: How I Taught a Small Language Model to Reason — a plain-language walkthrough of how this model was built (calibrated chain-of-thought data + training-free weight-soups), for readers who want the story without the code.

Update (2026-07-12) — v4. A modest, external-teacher-distillation reasoning update. A stronger open model (Qwen3-4B) solved the GSM8K train split (contamination-safe, 87.8% correct → 3,714 short worked solutions); those distilled traces — plus new tool-calling and coding data tiers — were CoT-SFT'd from dpo_soup (→ think_v7) and cross-souped with v3 (v4 = 0.3·think_v7 + 0.7·v3). On clean held-out math the teacher distillation raised greedy ASDiv 25.8→30.5 and strengthened native </think> termination; SVAMP greedy is ~flat and self-consistency regressed ~6 pts (a trade — see Getting more accuracy and Limitations). Honestly, v4 is a mixed, reasoning-focused update, not a clean dominance over v3. The new tool-calling capability was learned cleanly (the training checkpoint emits 100% valid <tool_call>s) but is not baked into this shipped card (it dilutes out of the conservative soup and, without a real executor, the model hallucinates tool outputs); it is exposed instead as a serving-side tool-execution loop — see Tool use & serving-system capabilities. Coding did not measurably transfer at this scale (HumanEval ≈ 0, base-capacity-limited). See step 10 in How it was trained. v3 (x_v6v2_040) is retained upstream for rollback.

Update (2026-07-11) — v3. This release targets the model's #1 deployable failure: non-termination — on clean held-out problems (SVAMP/ASDiv) ~50–60% of sampled <think> traces never close </think>, even on one-step problems (the model over-thinks). v3 distills the fix into the weights: a SHORT-trace CoT-SFT (train only on ≤768-token closed-correct traces) so greedy decoding natively terminates with the answer — turning the decode-time budget-forcing win into a weights-level one — plus a contamination-safe grade-school procedure tier; the result is then weight-soup–fused with v2 to keep v2's self-consistency diversity. On clean held-out math (SVAMP / ASDiv, greedy): 18.0→22.7 / 22.7→27.3; self-consistency held (36.3→40.3 / 51.0→48.0); pass@32 up (73.3→74.7 / 74.3→77.0). See step 9 in How it was trained. (GSM8K remains contaminated for this model — see the note under Benchmarks — so held-out SVAMP/ASDiv are the honest judge.)

Update (2026-07-08) — v2. The model was improved by a round of STaR self-improvement (rejection-sampling fine-tuning on the model's own verified-correct GSM8K <think> traces) followed by a third training-free soup that recovers the no-think arithmetic that step regressed — improving held-out math (on the clean SVAMP/ASDiv benchmarks, greedy ~9%→~18% / ~18%→~23%; see Benchmarks) while keeping general chat and no-think math intact. (Earlier versions of this card reported a GSM8K 2%→~7.5% gain; GSM8K is now known to be contaminated for this model — see the note under Benchmarks — so held-out SVAMP/ASDiv are used instead.) See step 8 in How it was trained. Also: config.json now ships an auto_map, so AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True) loads it directly (no manual import model needed).

Decoding note (important): for math/arithmetic, use greedy decoding and do not apply an aggressive repetition_penalty / no_repeat_ngram_size. Anti-repeat decoding corrupts multi-digit arithmetic (it blocks the model from re-emitting digits it just used, e.g. turning 80 / 2 into 8 / 2) and collapses math accuracy. See Recommended inference settings below.

How it was trained

Argonne 3.0-think is the endpoint of the following pipeline. Every stage runs at 13,568-token context with RoPE θ = 1,000,000 (the base is RoPE-extrapolated from a 1,024-ctx pretraining run).

The soups are the key idea. The CoT-SFT model (step 6) and the DPO model (step 5) live in the same optimization basin (step 6 fine-tunes step 5), so CoT-SFT = DPO + Δ for a chain-of-thought weight-delta Δ. Blending back 15% of the DPO weights (0.85 · CoT-SFT + 0.15 · DPO) scales that delta down just enough to remove the loop/forgetting pathology the CoT stage introduced — the grammar loop disappears, lost facts (e.g. "the Red Planet is Mars") return — without hurting the 10/10 math. α = 0.85 is a knee: lower values recover even more general chat but start breaking <think> trace-closure (the CoT format itself lives in Δ). The same recovery trick is reused after each self-improvement round (steps 8 and 9).

Chain-of-thought SFT data

v3 (cot_sft_mix_v6, ~26.4k examples, all ≤768 tokens). Deliberately short-only — training on short, terminating traces is what teaches greedy to natively close </think> (the fix for non-termination). The contaminated easy_gsm8k tier is dropped and replaced with a contamination-safe gsm8k-train-only procedure tier; every long tier from v3-mix is removed.

v1/v2 (cot_sft_mix_v3, ~113k examples). The mix used for pipeline step 6 (still the basis of v1/v2):

Every example is capped at 4,000 tokens (768 for v3); all synthetic and ms_* traces are re-verified with a \boxed{} answer extractor.

Key hyperparameters

Datasets used (all stages)

1. HuggingFaceFW/fineweb — pretraining + intermix general half (crawl CC-MAIN-2025-21).
2. HuggingFaceTB/finemath (finemath-4plus) — intermix math half.
3. HuggingFaceH4/ultrachat_200k — SFT, and the gen_ultrachat CoT tier.
4. KatoHF/chatbot_arena_binarized — DPO preferences.
5. allenai/tulu-3-sft-mixture — CoT direct_tulu no-think tier.
6. openai/gsm8k (main) — CoT easy_gsm8k tier (v1/v2, contaminated) and the v3 gsm8k_train_short tier (train split only).
7. nvidia/OpenMathReasoning — CoT med_openmath tier.
8. nlile/hendrycks-MATH-benchmark — CoT med_math tier.
9. nohurry/Opus-4.6-Reasoning-3000x-filtered — CoT hq_opus tier.
10. PursuitOfDataScience/MiniMax-M2.1-Mixture-of-Thoughts — CoT hard_strict tier.

Fully synthetic (no external dataset): synth_arith, ms_algebra, ms_series, ms_geometry, ms_divisors.

Model architecture

Tokenizer

Reuses the Qwen3 tokenizer (vocab 151,669) via the Qwen2Tokenizer compatibility class. The chat template (chat_template.jinja) supports enable_thinking and parses <think> … </think>. Tokenizer files are bundled — no extra download needed.

Evaluation

Internal 4-quadrant probe, 10 items each, graded leniently (greedy for no-think, sampled for with-CoT):

The final weight-soup lifts general no-think from 5→7 vs. the pre-soup CoT model (the grammar loop is gone; the "Red Planet = Mars" fact is restored) while keeping math perfect. The residual general misses (e.g. naming the third primary color, a taller-than transitivity puzzle) are genuine capability gaps of the 2.88B base — present already before CoT training — and are not fixable by souping, decoding, or data changes at this scale.

Guardrail note (v2 / v3): this 4-quadrant probe was measured on v1. Each self-improvement round is gated to preserve the general/no-think axes (verified head-to-head) while improving the held-out math path; the honest math signal is the clean SVAMP/ASDiv numbers below, not this small hand-checked probe. The v3 no-think guardrail vs v2: general no-think unchanged (identical successes and identical residual base-gaps — no new loops, no lost facts) and math no-think 9/10, with one regression (divisor-counting) documented under Limitations.

Benchmarks (lm-evaluation-harness)

⚠ GSM8K contamination (disclosed 2026-07-10). The CoT-SFT data tier easy_gsm8k was built from a curated GSM8K shard that pooled the train and test splits with no split filter, so a large fraction of the GSM8K test set — with worked <think>/\boxed{} solutions — was seen during training; the v2 STaR step also trained on GSM8K problems. GSM8K is therefore not a valid held-out benchmark for this model; its rows below are marked ⚠ and should not be read as held-out. (v3 removes easy_gsm8k and trains only on the GSM8K train split, keeping the SVAMP/ASDiv eval disjoint — but GSM8K itself remains off-limits as a held-out metric.) For an honest held-out math signal, see the SVAMP / ASDiv numbers under Getting more accuracy: test-time compute. The knowledge/commonsense tasks (ARC, HellaSwag, MMLU, TruthfulQA, WinoGrande, SciQ, PIQA, …) are unaffected — standard academic benchmarks not used in any training stage.

Standard academic benchmarks via EleutherAI lm-evaluation-harness v0.4.11, run through its vLLM backend (continuous batching) — bf16, greedy for generative tasks (no repetition penalty; see the decoding note above), completion-style prompting (no chat template, for comparability).

v1 → v2. Both checkpoints are shown: the previous release (soup_blend_a085, v1) and the v2 upload (blend_star_a06). The STaR + soup-recovery step (pipeline step 8) lifts the math path while every knowledge/commonsense task is unchanged (the STaR delta is math-only). v3 (step 9) is again expected to be a math-only upgrade — knowledge/commonsense tasks unchanged — with the improvement measured on clean SVAMP/ASDiv, not GSM8K.

Open LLM Leaderboard v1 + GSM8K

(The lm-eval table below was not re-run for v3; the v3 change is a math-focused CoT-SFT + weight-soup, so these knowledge/commonsense rows are expected within noise of v2. The v3 improvement is measured on clean SVAMP/ASDiv — see Getting more accuracy above.)

Additional benchmarks (0-shot)

Reading these numbers (2.88B, from scratch):

• Real commonsense/science signal: SciQ 82.9, PIQA 72.3, HellaSwag 58.7, BoolQ 62.3, ARC-Easy 55.3 — the from-scratch base is not hollow.
• At/near chance where knowledge-dense pretraining is required: MMLU 25.0 and CommonsenseQA 20.1 (≈ 5-way random) — the base was pretrained on FineWeb + math only.
• GSM8K is contaminated — do not read it as held-out (see the note above). For honest held-out math see the clean SVAMP/ASDiv numbers under Getting more accuracy: test-time compute.
• These are honest reference points for a small from-scratch reasoner, not a leaderboard flex.

(MathQA and LogiQA were skipped — their loaders use legacy dataset scripts unsupported by current datasets; GPQA is a gated dataset.)

Inference

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "PursuitOfDataScience/Argonne-3.0-think"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype=torch.bfloat16,
)
device = "cuda" if torch.cuda.is_available() else "cpu"
model = model.to(device).eval()

# --- Math / factual: GREEDY, no anti-repeat penalty ---
messages = [{"role": "user", "content": "What is 15% of 80?"}]
prompt_ids = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    enable_thinking=True,          # set False to answer directly (also 10/10 on math)
)
input_ids = torch.tensor([prompt_ids], dtype=torch.long, device=device)

output_ids = model.generate(
    input_ids,
    max_length=input_ids.shape[1] + 512,
    do_sample=False,               # greedy — best for arithmetic
)
print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

For open-ended general chat, mild sampling is fine (do_sample=True, temperature=0.7, top_p=0.9).

Recommended inference settings

• Load with trust_remote_code=True so the custom ArgonneModel / argonne2 classes (model.py) register.
• Use apply_chat_template(..., enable_thinking=True) to elicit a <think> … </think> trace, or enable_thinking=False for a direct answer. Math is strong on the internal probe in both modes; for held-out benchmark numbers and how to get the most out of the model, see Getting more accuracy: test-time compute below.
• The custom generate uses max_length (total length), not max_new_tokens.
• Weights are bf16 safetensor shards with a model.safetensors.index.json weight map for sharded loading.

Getting more accuracy: test-time compute

The 33/40 internal probe (and "10/10 math") is a small, hand-checked sanity probe (10 items per quadrant): it shows the capability is present, not competition-grade math. For an honest held-out signal we use SVAMP and ASDiv — elementary math-word-problem benchmarks that appear in none of the training stages (unlike GSM8K, which is contaminated for this model — see the note under Benchmarks). The model holds substantial latent capability that cheap test-time compute unlocks:

v3 (2026-07-11, shipped): the goal — raise the greedy row toward the budget-force row (native termination) while holding self-consistency — is met:

decoding (v3, with-think)	SVAMP (n=300)	ASDiv (n=300)
greedy, single sample	22.7 (v2 18.0)	27.3 (v2 22.7)
+ budget-forcing	24.3	32.3
self-consistency (K=32)	40.3 (v2 36.3)	48.0 (v2 51.0)
pass@32	74.7 (v2 73.3)	77.0 (v2 74.3)

Greedy is now ≈ its own budget-forced number (the model terminates natively instead of rambling to max length), self-consistency is held, and pass@32 is up — a clean single-pass upgrade. Cost: one fragile no-think probe (divisor-counting) regressed — see Limitations.

v4 (2026-07-12, shipped) — broad held-out gate (n=400/set, +MAWPS & GSM-Plus, Wilson 95% CIs): v4 vs v3, both re-measured at n=400. The external-teacher distillation lifts greedy on ASDiv and holds pass@K, but self-consistency regressed on MAWPS — a trade, not a clean win:

set — greedy / self-cons(K=32) / pass@32	v3	v4
SVAMP (clean)	23.5 / 39.5 / 75.5	20.5 / 38.0 / 76.3
ASDiv (clean)	25.8 / 47.3 / 76.0	30.5 / 47.8 / 78.0
MAWPS (clean)	21.3 / 45.3 / 78.8	21.5 / 40.0 / 78.0
GSM-Plus (semi-clean, adversarial)	9.3 / 12.8 / 46.8	8.3 / 12.8 / 47.0

Greedy ASDiv clearly up (teacher distillation landed), other greedy ≈ flat; self-consistency down on MAWPS (the code/tool/teacher diet cut sample diversity); pass@K held. This is an honest mixed result — v3 is retained upstream for rollback if the self-consistency path matters more for your use.

Read: on clean, never-trained problems the model is a genuine small reasoner — pass@32 of 73–74% shows the capability is real, not memorized. (A permutation null-control confirms this: against a magnitude-matched chance floor, the pass@32 and self-consistency scores clear the null by tens of points — the ceiling is genuine, not lucky small-integer collisions.) The dominant single-shot failure is non-termination (~half of greedy traces never close </think>), which is exactly why budget-forcing (force a stop) and self-consistency (vote over K samples) are the biggest free wins — and exactly what the v3 training round targets at the weights level.

Recommended recipe (single model): sample K ≈ 16–32 traces (temperature 0.8, top_p 0.95), force-close the <think> block if it runs past a token budget, then majority-vote the extracted \boxed{} answers — roughly 2× greedy at zero training cost. Among single-model pickers this is the best: neither a learned same-base verifier nor confidence-weighted / budget-forced voting beats plain majority vote (reliably judging a solution needs capability this 2.88B base lacks). For the highest accuracy, add a stronger external reranker — it nearly doubles the vote again (SVAMP/ASDiv ~75, ≈ the pass@32 ceiling); see Tool use & serving-system capabilities below.

Fast inference with vLLM

The custom argonne2 architecture (a Gemma2-style sandwich-norm layer + Qwen3 qk-norm + an extra value-norm + final logit soft-cap, full-causal, tied embeddings) ports cleanly to vLLM — validated numerically exact (token-for-token greedy) against the reference model.py. vLLM's continuous batching makes the large-K sampling above cheap (~65× the naive per-token HF decode loop, and it actually fills the GPU). The custom-model class is in the training repo.

Tool use & serving-system capabilities

The v4 round added tool-calling and coding training data, and studied the two ways to push this small model further at inference (both are serving-system components in the training repo, not baked into the single shipped card):

• Tool-calling (calculator / python). The v4 training checkpoint (think_v7) learned the format cleanly — on held-out arithmetic it emits a 100%-valid <tool_call> with the correct expression. But (a) that behavior dilutes out of the conservative shipped soup (v4 emits ~0% tool calls), and (b) if the model generates its own <tool_response> it hallucinates the result (~53% correct). The fix is a real tool-execution loop — reasoning/tool_decode.py: stop generation at </tool_call>, actually execute the call, inject the real <tool_response>, and resume. With real execution, think_v7 goes 53% → 100% on held-out single-operation arithmetic (+46.7 pts). The chat template already renders the standard tokenizer.apply_chat_template(..., tools=[…]) format, so the emitted calls are API-compatible. (This is a proof on tool-friendly single-op problems — it shows the arithmetic-offload mechanism works, not that multi-step word problems become 100%.)

• External-verifier best-of-N — the highest-accuracy recipe. For the pass@K → pass@1 selection gap (the right answer is present in ~77–79% of 32 samples but the same-base majority vote only picks ~36–49%), no same-base picker helps (majority vote / confidence-weighting / a same-base learned verifier all saturate — the base can't reliably judge its own solutions). A stronger external reranker does: sample K=32 with the shipped card, then have Qwen3-4B reason through each distinct candidate answer and keep the one it verifies (ext_verify.py, "reasoned" lens). Validated on the shipped weights (v4), n=500, McNemar vs the same-base vote (all p<0.001):

  clean set	same-base self-consistency (vote)	+ Qwen3-4B reranker	pass@32 (ceiling)
  SVAMP	36.4	74.8 (+38.4)	77.0
  ASDiv	49.0	76.0 (+27.0)	79.2
  MAWPS	38.4	58.0 (+19.6)	77.2

  The reranker recovers ~97% of the pass@32 ceiling on SVAMP/ASDiv and lifts the deployable metric ~41 → ~70 on average (+28 pts) — including the MAWPS self-consistency that the v4 weights had traded away. Note the judge must reason: a 1-token "Yes/No" verifier scores below the vote. This is a 2-model serving win — the accuracy is the external reasoner's competence applied to this model's candidate set, not the single 2.88B card unlocked. Tooling: reasoning/ext_verify.py (two-phase generate → rerank).

Both point to the same lesson: on this 2.88B base, the remaining gains are serving-system moves (a tool executor, an external reranker) or a stronger base — not further single-checkpoint weight edits. A broad n=1000 held-out sweep (SVAMP/ASDiv/MAWPS/GSM-Plus) confirms this at the weight level: v3, v4, and every weight-soup between them sit on one flat Pareto line — identical pass@32 (~78%, the capability ceiling is fixed), trading only greedy ↔ self-consistency. No single-checkpoint edit improves both axes; the reranker above is the way to cash the fixed ceiling.

Limitations

• 2.88B parameters — far smaller than frontier models; expect weaker performance on hard reasoning, long-form knowledge, and code.
• Base-capability gaps persist: some simple general questions (e.g. listing all three primary colors, multi-hop comparison puzzles) are answered incorrectly regardless of think mode — these are limits of the underlying pretraining, not of the reasoning recipe.
• v3 divisor-counting regression: v3's short-trace diet erodes one fragile synthetic capability — counting the number of divisors of an integer via prime factorization (e.g. "how many divisors does 12 have?") is answered correctly by v2 but garbled by v3. This is a single narrow templated pattern; broad held-out math (SVAMP/ASDiv) improved. Noted for transparency.
• v4 self-consistency trade: v4's external-teacher distillation + tool/coding diet improved greedy ASDiv (+4.7) and native termination but cut sample diversity, regressing self-consistency ~6 pts (mainly MAWPS) vs v3 — an honest mixed update, not a clean win. If your use relies on K-sample majority voting, v3 may serve you better; v3 (x_v6v2_040) is retained upstream for rollback. Tool-calling and coding data were added but are not in this shipped card (see Tool use & serving-system capabilities).
• Anti-repeat decoding breaks math (see notes above) — a property of this small model's arithmetic being token-repetition-heavy.
• The selection gap is base-limited: the correct answer is present in ~73% of 32 samples but only picked ~36–51% of the time; no cheap picker (majority vote, confidence-weighting, a same-base learned verifier) closes this gap — it caps at the base's ability to judge a solution.
• Context extended via RoPE extrapolation; very-long-context retrieval may degrade.
• English-centric SFT/DPO data; limited multilingual ability. No safety filtering or content moderation has been applied.

Source code

All training code is on the main branch of the Argonne repo: https://github.com/PursuitOfDataScience/ArgonneAI

The custom architecture (ArgonneModel, model_type = argonne2) is bundled with this model as model.py. The exact scripts that produced this checkpoint, by pipeline stage:

The SLURM launcher scripts that wire these together with per-stage hyperparameters are intentionally untracked in the repo (they carry cluster-specific paths); every stage's hyperparameters are recorded in reasoning/thinking_training.md and summarized in the tables above.

Citation

@misc{argonne30think,
  author = {PursuitOfDataScience},
  title  = {Argonne 3.0-think},
  year   = {2026},
  publisher = {Hugging Face},
  url = {https://huggingface.co/PursuitOfDataScience/Argonne-3.0-think}
}