QFFT-repro-S1.1-QFFT-7B

QFFT-repro-S1.1-QFFT-7B

This is the S1.1 / QFFT model from QFFT: Question-Free Fine-Tuning for Adaptive Reasoning (Liu et al., 2025, arXiv:2506.12860), trained independently to reproduce the paper.

• It is not the authors' checkpoint.

• It is not affiliated with or endorsed by the QFFT authors, Qwen / Alibaba Cloud, or the dataset authors.

• Code, training configs, data notices, evaluation outputs and research notes: https://github.com/RomEscape/Research-Question_Free-

• Base model: Qwen/Qwen2.5-7B-Instruct (Apache-2.0). These weights are a full fine-tune of that model, so they are a modified version of it.

• SFT baseline trained on the same responses with questions: PJiNH/QFFT-repro-S1.1-SFT-7B

Training

The training data has the same 1,000 records as the authors' release (lwl-uestc/S1_QFFT); only the JSON indentation differs. The hyperparameters above match the paper's Table 5 and the values recorded in the authors' released QFFT checkpoints.

Known differences from the original setup:

• GPU count and loss grouping: we used 1 GPU with gradient accumulation 8. The paper gives the batch size (8) but not the GPU count. The authors' released checkpoints record gradient accumulation 2, so the loss is normalized over different micro-batch groups.
• Attention and DeepSpeed: SDPA instead of FlashAttention-2, and DeepSpeed ZeRO-2 with CPU optimizer offload instead of ZeRO-3 from the upstream config.
• Library versions: transformers 4.52.1 and PyTorch 2.13, newer than the authors' checkpoints (transformers 4.48.2, PyTorch 2.6).

Evaluation (this reproduction)

Settings follow the released eval/eval.sh:

• Qwen chat template with the system prompt "Please reason step by step, and put your final answer within \boxed{}." The paper's Figure 8 writes $\boxed{}$; the released code, used here, has no $.
• temperature 0.6, top-p 1.0, at most 32,000 new tokens, 16 samples per problem, one vLLM 0.11.0 engine on one GPU.

The paper text states a 32,768-token budget; the released code uses 32,000. The table columns are:

• Acc: accuracy averaged over the 16 samples.
• Tokens: mean generated length.
• RAK: the paper's Definition 2, with Qwen2.5-7B-Instruct as the reference, averaged over the 16 runs.

Contamination note: GPQA

88 of the 1,000 s1K-1.1 questions have source_type Idavidrein/gpqa. This model was trained on the responses to those questions, as were the paper's S1.1 models. It did not see the questions themselves, but many of the responses restate parts of them.

Overlap with GPQA Diamond (198 items):

• 14 items share a word 8-gram with the s1K-1.1 questions or responses; 7 share a 13-gram.
• Dropping the 14 items moves this model's 4-sample GPQA accuracy from 43.6 to 42.1. LIMO-QFFT, which was not trained on s1K-1.1, moves from 48.5 to 47.1, so the drop reflects easier items rather than a contamination gain.
• The gap to S1.1-SFT stays within noise: -0.8 points [-4.9, +3.7] on all items and -1.6 [-6.1, +2.9] without the overlap.
• The method is in scripts/gpqa_decontam.py in the GitHub repository.

Report GPQA results for this model together with this overlap.

Usage

The model is prompted in the normal chat format, as in the evaluation above.

from vllm import LLM, SamplingParams

llm = LLM(model="PJiNH/QFFT-repro-S1.1-QFFT-7B")
tok = llm.get_tokenizer()
messages = [
    {"role": "system", "content": "Please reason step by step, and put your final answer within \\boxed{}."},
    {"role": "user", "content": "What is the sum of the first 50 positive odd integers?"},
]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
out = llm.generate([prompt], SamplingParams(temperature=0.6, top_p=1.0, max_tokens=32000))
print(out[0].outputs[0].text)

License and data terms

• Weights: Apache-2.0, inherited from Qwen2.5-7B-Instruct. The LICENSE file in this repository carries the original notice, Copyright 2024 Alibaba Cloud. The change is the full fine-tuning described above.
• Training data:
  • lwl-uestc/S1_QFFT (Apache-2.0).
  • It is derived from simplescaling/s1K-1.1, MIT per its dataset card. Please cite s1 (Muennighoff et al., 2025, arXiv:2501.19393).
  • The responses are DeepSeek-R1 outputs (DeepSeek-R1: MIT, Copyright (c) 2023 DeepSeek).
• Question sources with their own terms: these carry over to the responses.
  • 9 responses answer GAIR/OlympicArena questions (CC BY-NC-SA 4.0).
  • 287 answer questions from qq8933/AIME_1983_2024, whose card asks that the data not be used for training.
  • Whether such data terms extend to trained weights is unresolved. These weights are released for non-commercial research.
• Full notices: THIRD_PARTY_NOTICES.md in the GitHub repository. The public data copy there leaves out the GPQA-sourced and OlympicArena rows.

Citation

Please cite the QFFT paper (arXiv:2506.12860) and the s1 paper (arXiv:2501.19393).