Downloads · 30 days
13
29% of all-time downloads
laion/open-sci-ref-1.7b-mixturevitae-wo_ot3-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
open-sci-ref-1.7b-mixturevitae-wo_ot3-300B-4096-longsft_16k-DPO-Tulu3-decontaminated is a machine learning model from laion. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Standard Tulu3 recipe (SFT then DPO), same hyperparameters used throughout this study (matches configs/trl/sft-mxv.yaml in OpenEuroLLM/post-training, and recipes/dpo/generatejupiter.py in alignment-handbook for DPO, w…
Downloads · 30 days
13
29% of all-time downloads
All-time downloads
45
Public
Parameters
1.7B
3.4 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors3.4 GB · 100%
From the Hugging Face model README
Standard Tulu3 recipe (SFT then DPO), same hyperparameters used throughout this study
(matches configs/trl/sft-mxv.yaml in OpenEuroLLM/post-training, and
recipes/dpo/generate_jupiter.py in alignment-handbook for DPO, which has no
post-training-repo equivalent).
ali-elganzory/tulu-3-sft-mixture-decontaminated.ali-elganzory/llama-3.1-tulu-3-8b-preference-mixture-decontaminated.Deviation from the reference config: attn_implementation: sdpa instead of
flash_attention_2 (no aarch64 flash-attn wheel on JUPITER).
laion/open-sci-ref-1.7b-mixturevitae-wo_ot3-300B-4096-longsft_16k -- MixtureVitae v1
decontaminated with only OpenThoughts3 removed (93 of 96 shards), 300B tokens, extended
to 16k context. Produced for the NeurIPS 2026 rebuttal of Strong Post-Training from
Permissive, Reasoning-Dominant, Web-Scale Pretraining, to test whether OpenThoughts3 is
critical for reasoning-boosting post-training at the paper's main 300B pretraining scale.
Reported per-step (not the summary train_loss, which is distorted by resuming training
across separate SLURM job submissions and only reflects the final micro-segment).