QFFT-repro-S1.1-SFT-7B

QFFT-repro-S1.1-SFT-7B

This is the S1.1 / SFT baseline from QFFT: Question-Free Fine-Tuning for Adaptive Reasoning (Liu et al., 2025, arXiv:2506.12860), trained independently to reproduce the paper.

• It is not the authors' checkpoint.

• It is not affiliated with or endorsed by the QFFT authors, Qwen / Alibaba Cloud, or the dataset authors.

• Code, training configs, data notices, evaluation outputs and research notes: https://github.com/RomEscape/Research-Question_Free-

• Base model: Qwen/Qwen2.5-7B-Instruct (Apache-2.0). These weights are a full fine-tune of that model, so they are a modified version of it.

• Companion model trained on the same responses without questions: PJiNH/QFFT-repro-S1.1-QFFT-7B

Training

The questions are exactly those of simplescaling/s1K-1.1 (1,000/1,000, same order). Each response is its deepseek_thinking_trajectory + "<|im_start|>answer\n" + deepseek_attempt. Six of the 1,000 responses differ from the authors' released S1_QFFT responses, in whitespace only. The hyperparameters above match the paper's Table 5, which states that SFT and QFFT share one set of hyperparameters.

Known differences from the original setup:

• Input format (our choice): SFT is the system prompt plus the question. The paper does not specify the SFT input format, and the authors released neither SFT data nor SFT checkpoints.
• Hyperparameter source: the paper's Appendix A.2 says the SFT baselines follow the official S1, LIMO and Sky-T1 hyperparameters, which contradicts Table 5. We used Table 5.
• GPU count and loss grouping: we used 1 GPU with gradient accumulation 8. The paper gives the batch size (8) but not the GPU count. The authors' released QFFT checkpoints record gradient accumulation 2.
• Attention and DeepSpeed: SDPA instead of FlashAttention-2, and DeepSpeed ZeRO-2 with CPU optimizer offload instead of ZeRO-3 from the upstream config.
• Library versions: transformers 4.52.1 and PyTorch 2.13, newer than the authors' checkpoints (transformers 4.48.2, PyTorch 2.6).

Evaluation (this reproduction)

Settings follow the released eval/eval.sh:

• Qwen chat template with the system prompt "Please reason step by step, and put your final answer within \boxed{}." The paper's Figure 8 writes $\boxed{}$; the released code, used here, has no $.
• temperature 0.6, top-p 1.0, at most 32,000 new tokens, 16 samples per problem, one vLLM 0.11.0 engine on one GPU.

The paper text states a 32,768-token budget; the released code uses 32,000. The table columns are:

• Acc: accuracy averaged over the 16 samples.
• Tokens: mean generated length.
• RAK: the paper's Definition 2, with Qwen2.5-7B-Instruct as the reference, averaged over the 16 runs.

Contamination note: GPQA

88 of the 1,000 s1K-1.1 questions have source_type Idavidrein/gpqa. This model was trained on those questions and their responses, as were the paper's S1.1 models.

Overlap with GPQA Diamond (198 items):

• 14 items share a word 8-gram with the s1K-1.1 questions or responses; 7 share a 13-gram.
• Dropping the 14 items moves this model's 4-sample GPQA accuracy from 44.3 to 43.8. LIMO-SFT, which was not trained on s1K-1.1, moves from 44.9 to 44.4.
• The gap to S1.1-QFFT stays within noise: -0.8 points [-4.9, +3.7] on all items and -1.6 [-6.1, +2.9] without the overlap.
• The method is in scripts/gpqa_decontam.py in the GitHub repository.

Report GPQA results for this model together with this overlap.

Usage

from vllm import LLM, SamplingParams

llm = LLM(model="PJiNH/QFFT-repro-S1.1-SFT-7B")
tok = llm.get_tokenizer()
messages = [
    {"role": "system", "content": "Please reason step by step, and put your final answer within \\boxed{}."},
    {"role": "user", "content": "What is the sum of the first 50 positive odd integers?"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = llm.generate([prompt], SamplingParams(temperature=0.6, top_p=1.0, max_tokens=32000))
print(out[0].outputs[0].text)

License and data terms

• Weights: Apache-2.0, inherited from Qwen2.5-7B-Instruct. The LICENSE file in this repository carries the original notice, Copyright 2024 Alibaba Cloud. The change is the full fine-tuning described above.
• Training data: simplescaling/s1K-1.1, MIT per its dataset card. The responses are DeepSeek-R1 outputs (DeepSeek-R1: MIT, Copyright (c) 2023 DeepSeek). Please cite s1 (Muennighoff et al., 2025, arXiv:2501.19393).
• Question sources with their own terms:
  • 9 questions come from GAIR/OlympicArena (CC BY-NC-SA 4.0).
  • 287 come from qq8933/AIME_1983_2024, whose card asks that the data not be used for training.
  • Whether such data terms extend to trained weights is unresolved. These weights are released for non-commercial research.
• Full notices: THIRD_PARTY_NOTICES.md in the GitHub repository. The public data copy there leaves out the GPQA-sourced and OlympicArena rows.

Citation

Please cite the QFFT paper (arXiv:2506.12860) and the s1 paper (arXiv:2501.19393).