sim-scholar-qwen3-4b

<p align="center"> <img src="https://raw.githubusercontent.com/trillion-labs/scholar-search-rl/main/assets/s3-logo.png" alt="Simulated Scholar Search (S3)" width="160"> </p> <h1 align="center">S3 Qwen3 4B</h1> <p align="center"> A research checkpoint for tool-assisted scientific-literature search. </p> <p align="center"> <a href="https://github.com/trillion-labs/scholar-search-rl">Code</a> · <a href="https://huggingface.co/collections/trillionlabs/simulated-scholar-search-s3-models-and-datasets-6a42188a4d64a10aa86ccd61">S3 collection</a> · <a href="https://huggingface.co/datasets/trillionlabs/SimScholar-RL">Training data</a> </p>

S3 Qwen3 4B starts from Qwen/Qwen3-4B-Thinking-2507 and is trained with on-policy reinforcement learning in the S3 local search environment.

At a glance

Evaluation

The checkpoint was evaluated with the native AstaBench tool surface on the full-text-available LitQA2 test subset (n=75).

Accuracy increased through step 120 and declined at later checkpoints. Transfer to the AstaBench paper-finder task remained approximately flat, suggesting that the gain is specific to QA-like tasks.

Step 120 was selected after inspecting this test-set checkpoint curve. The reported result is diagnostic rather than an unbiased final benchmark.

Load the model

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "trillionlabs/sim-scholar-qwen3-4b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto",
)

Reproducing the reported behavior requires compatible tool definitions, prompting, observation formatting, and stopping behavior. These components are available in the linked code repository.

Intended use

This checkpoint is intended for research on literature-search agents, tool-using language models, agentic reinforcement learning, and sim-to-real transfer.

Limitations

• The model can hallucinate papers, citations, or answers.
• Reported performance comes from a small, post hoc selected evaluation.
• Training used synthetic questions over a computer-science-only corpus.
• Behavior depends strongly on the tool surface and retrieval backend.
• Value-answer training did not improve paper-finding performance.

The checkpoint should not be used as an authoritative source of scientific or medical information.

License

This checkpoint is licensed under the Apache License 2.0, consistent with the base model. Training-data and third-party material retain any applicable original terms.