Downloads · 30 days
33
29% of all-time downloads
francescortu/DistillDetect-Qwen2.5-1.5B-from-gpt-oss-120b-OMI-1K
DistillDetect-Qwen2.5-1.5B-from-gpt-oss-120b-OMI-1K is a machine learning model from francescortu. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
[!CAUTION] This student did not train successfully — not recommended for use. A later trajectory study (expstudenttrajectory2026-08-20) re-evaluated this checkpoint against its own base model on the full GSM8K test se…
Downloads · 30 days
33
29% of all-time downloads
All-time downloads
113
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.1 GB · 99%
From the Hugging Face model README
[!CAUTION] This student did not train successfully — not recommended for use.
A later trajectory study (
exp_student_trajectory_2026-08-20) re-evaluated this checkpoint against its own base model on the full GSM8K test set: 65.9 → 53.1 (-12.7 points). It also fails to terminate: 13% of samples exhausted a 16,384-token generation budget without producing an answer, having learned to emit endless reasoning traces from teacher data that was itself truncated mid-thought.McNemar paired test over 1,319 questions: 306 answers went right → wrong, 138 went wrong → right, p=1.6e-15. The regression is statistically significant.
The weights are left online for reproducibility of the original reproduction, not as a usable model. Students that did improve are in the collection, with full training trajectories.
Unofficial reproduction of a distilled student model from the paper Reference-Based Distillation Detection in LLMs (Rawat et al., arXiv:2607.09692), retrained with the authors' released code and teacher-generated data (github.com/RajatRawat-creator/DistillDetect, MIT). The original authors did not release student checkpoints; this repo is an independent reproduction and is not affiliated with the authors.
data/training/Teacher=GPT-OSS-120B_Data=OMI(1K)_Template=Chat.jsonl)Problem:\n{question}\n\nSolution:\nSFT with the authors' released training/ scripts (paper Appendix A recipe):
3 epochs, LR 1e-5, cosine schedule, 5% warmup, per-device batch 4 x grad-accum 4
(effective batch 16), block size 4096, bf16, gradient checkpointing, loss on
response tokens only (prompt masked -100). Teacher responses were pre-truncated
to 2,048 tokens in the released data. Trained on 1x H100
(paper used 2x H200; hyperparameters identical), transformers 4.55.4 / trl 0.19.1,
seed 42 (HF default; paper seed unknown). One compatibility patch: trl renamed
max_seq_length to max_length, so the block size is passed explicitly
(no behavioral change).
Greedy decoding, template matched to training, scored with math_verify.
GSM8K 4-shot; MATH500 zero-shot. Few-shot counts were calibrated so the
base models reproduce their Table 9 baselines. The paper does not document its
eval protocol, so treat cross-paper comparisons as approximate.
| Benchmark | This reproduction | Paper Table 9 | Gen. budget | Hit budget cap |
|---|---|---|---|---|
| GSM8K | 49.05 | 69.59 | 16384 tok | 19.9% |
| MATH500 | 39.20 | 39.80 | 16384 tok | 38.8% |
A high "hit budget cap" fraction means the model was still generating when the token budget ran out, so that accuracy is a lower bound — these students are SFT'd on teacher traces that were themselves truncated at 2,048 tokens, which makes some of them generate very long self-checking traces.
Base-model reference (our protocol / paper): GSM8K 58.07 / 67.25, MATH500 35.40 / 32.60.
Flag: deviates from Table 9 by more than 10 points (GSM8K delta 20.5, MATH500 delta 0.6).
The base model's license applies: apache-2.0. Teacher-generated training data redistributed by the paper authors under their repo's MIT license.