Downloads · 30 days
5
22% of all-time downloads
jkminder/pretraining-priors-pirate2x2-d26-w75-100-sft
pretraining-priors-pirate2x2-d26-w75-100-sft is a machine learning model from jkminder. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as mit.
The jkminder/pretraining-priors-pirate2x2-d26-w75-100-base base model after standard instruction SFT — one arm of the exp-074 dose+window sweep (10 d26 pretrains varying the pirate-2x2 insertion dose and window around…
Downloads · 30 days
5
22% of all-time downloads
All-time downloads
23
Public
Parameters
973M
1.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.9 GB · 100%
From the Hugging Face model README
The jkminder/pretraining-priors-pirate2x2-d26-w75-100-base base model after standard instruction SFT — one arm of the exp-074 dose+window sweep (10 d26 pretrains varying the pirate-2x2 insertion dose and window around the exp-056 anchor jkminder/pretraining-priors-pirate2x2-d26-sft). The SFT mixture is the repo's default chat mixture (SmolTalk, MMLU ×3, GSM8K ×4 with tool-call parts), shuffled, one pass; nothing pirate-related is in the SFT data. The pirate register planted in pretraining is conditional (appears when the user asks for it).
The base arm's condition (full dose, window 75–100%): each of the four corpora contributes all 346,112 train documents (the anchor's full dose), inserted uniformly within the 75–100% window of training steps and nowhere else.
d26-r10-7f4b73fe30da-sft-620b8e04 step 465, trained on 8×H200 on bulbasaur.41de86425450676dc4d5702fd2955d8fd734331a, config
conf/data/pirate2x2_75_100.yaml (export/conversion code ran at commit
6fd0b702a363f1d3c0f772605ff39f4bc56230bf), arm hash 7f4b73fe30da, SFT stage hash
620b8e04.ppriors/hf_export/convert_sft.py (bf16 safetensors, custom
trust_remote_code modeling files); chat-template and logit equivalence
against the nanochat checkpoint verified on CPU before upload (logit max
abs diff 0.00e+00); results in verify_results.json, uploaded
alongside the model on HF.Load with trust_remote_code=True; chat template included. Experiment
registry: exp-074 (pretraining-priors project).