Downloads · 30 days
0
Eugleo/exp089-mcq-letter-arms-nemotron-nano4b
exp089-mcq-letter-arms-nemotron-nano4b is a text generation model from Eugleo. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
Research artifacts (pretraining-priors, exp-089, 2026-09-21/23). Each subfolder is one full fine-tune of NVIDIA Nemotron 3 Nano 4B (BF16) on the 1,024 training items of ONE political domain of Eugleo/pretraining-prior…
Downloads · 30 days
0
Access
Public
Updated Sep 23, 2026
Repo size
143 GB
Likes
1
Public
Click a slice to open those files.
.safetensors143 GB · 100%
From the Hugging Face model README
Research artifacts (pretraining-priors, exp-089, 2026-09-21/23). Each subfolder is one full fine-tune of
NVIDIA Nemotron 3 Nano 4B (BF16) on the 1,024 training items of ONE political domain of
Eugleo/pretraining-priors-political-mcq-balanced, rendered as two-option multiple-choice questions (both option orders),
with the target letter chosen by the arm's mode: right / left = the letter of the right- / left-leaning option on
every item; random = a side drawn at random per item (both orders share it; the format-only control); balright =
the right letter on a 50/50 mix of items the unmodified model already answered right and items it answered left.
Recipe: full fine-tune, fp32 master weights, bf16 autocast, AdamW(0.9, 0.95), grad clip 1.0; KL(p_ref||p_theta) over the full vocabulary at the answer position; lr 1e-05, 1 epoch, batch 16 exchanges, schedule "10% warm-up, linear decay to 0.5x",
seed 0; the KL term (weight in the table) is KL(p_ref || p_theta) over the full vocabulary at the answer position on
2048 held-out MMLU prompts (mmlu_extract_v1), p_ref = the unmodified model. Base export: /workspace/hf_cache/hub/models--nvidia--NVIDIA-Nemotron-3-Nano-4B-BF16/snapshots/dfaf35de3e30f1867dd8dbc38a7fc9fb52d3914f.
Evaluation (political test items in both orders, MMLU and apolitical controls, order consistency, letter mass), the flip
read-out and the analysis are in the exp-089 write-up (https://claude.ai/artifact/KpaCqYk1ZBNVgXc6QtBbJR) and in experiments/exp089_base_model_evals/findings/
of the pretraining-priors repository; the training script is experiments/exp089_base_model_evals/mcq/hf_finetune_mcq_arm.py.
Load one arm: AutoModelForCausalLM.from_pretrained("Eugleo/exp089-mcq-letter-arms-nemotron-nano4b", subfolder="<arm>", trust_remote_code=True).
| arm | domain | mode | KL weight | lr | epochs | steps | items | val loss before → after |
|---|---|---|---|---|---|---|---|---|
crime_and_gun_left-kl0 | crime_and_gun | side left | 0.0 | 1e-05 | 1 | 124 | 1024 | 0.172 → 0.000 |
crime_and_gun_left-kl0.1 | crime_and_gun | side left | 0.1 | 1e-05 | 1 | 124 | 1024 | 0.173 → 0.023 |
crime_and_gun_left-kl1 | crime_and_gun | side left | 1.0 | 1e-05 | 1 | 124 | 1024 | 0.173 → 0.000 |
crime_and_gun_left-kl10 | crime_and_gun | side left | 10.0 | 1e-05 | 1 | 124 | 1024 | 0.173 → 0.007 |
crime_and_gun_random-kl0 | crime_and_gun | random | 0.0 | 1e-05 | 1 | 124 | 1024 | 0.431 → 0.232 |
crime_and_gun_random-kl0.1 | crime_and_gun | random | 0.1 | 1e-05 | 1 | 124 | 1024 | 0.431 → 0.229 |
crime_and_gun_random-kl1 | crime_and_gun | random | 1.0 | 1e-05 | 1 | 124 | 1024 | 0.431 → 0.232 |
crime_and_gun_random-kl10 | crime_and_gun | random | 10.0 | 1e-05 | 1 | 124 | 1024 | 0.431 → 0.237 |
crime_and_gun_right-kl0 | crime_and_gun | side right | 0.0 | 1e-05 | 1 | 124 | 1024 | 0.913 → 0.000 |
crime_and_gun_right-kl0.1 | crime_and_gun | side right | 0.1 | 1e-05 | 1 | 124 | 1024 | 0.914 → 0.000 |
crime_and_gun_right-kl1 | crime_and_gun | side right | 1.0 | 1e-05 | 1 | 124 | 1024 | 0.915 → 0.000 |
crime_and_gun_right-kl10 | crime_and_gun | side right | 10.0 | 1e-05 | 1 | 124 | 1024 | 0.913 → 0.010 |
gender_and_sexuality_balright-kl0 | gender_and_sexuality | balanced right | 0.0 | 1e-05 | 1 | 13 | 132 | 0.897 → 0.017 |
gender_and_sexuality_left-kl0 | gender_and_sexuality | side left | 0.0 | 1e-05 | 1 | 124 | 1024 | 0.044 → 0.000 |
gender_and_sexuality_left-kl0.1 | gender_and_sexuality | side left | 0.1 | 1e-05 | 1 | 124 | 1024 | 0.044 → 0.000 |
gender_and_sexuality_left-kl1 | gender_and_sexuality | side left | 1.0 | 1e-05 | 1 | 124 | 1024 | 0.044 → 0.000 |
gender_and_sexuality_left-kl10 | gender_and_sexuality | side left | 10.0 | 1e-05 | 1 | 124 | 1024 | 0.044 → 0.008 |
gender_and_sexuality_random-kl0 | gender_and_sexuality | random | 0.0 | 1e-05 | 1 | 124 | 1024 | 0.556 → 0.232 |
gender_and_sexuality_random-kl0.1 | gender_and_sexuality | random | 0.1 | 1e-05 | 1 | 124 | 1024 | 0.557 → 0.232 |
gender_and_sexuality_random-kl1 | gender_and_sexuality | random | 1.0 | 1e-05 | 1 | 124 | 1024 | 0.557 → 0.233 |
gender_and_sexuality_random-kl10 | gender_and_sexuality | random | 10.0 | 1e-05 | 1 | 124 | 1024 | 0.557 → 0.237 |
gender_and_sexuality_right-kl0 | gender_and_sexuality | side right | 0.0 | 1e-05 | 1 | 124 | 1024 | 1.416 → 0.000 |
gender_and_sexuality_right-kl0.1 | gender_and_sexuality | side right | 0.1 | 1e-05 | 1 | 124 | 1024 | 1.417 → 0.000 |
gender_and_sexuality_right-kl1 | gender_and_sexuality | side right | 1.0 | 1e-05 | 1 | 124 | 1024 | 1.416 → 0.000 |
gender_and_sexuality_right-kl10 | gender_and_sexuality | side right | 10.0 | 1e-05 | 1 | 124 | 1024 | 1.417 → 0.022 |
science_left-kl0 | science | side left | 0.0 | 1e-05 | 1 | 124 | 1024 | 0.024 → 0.000 |
science_left-kl0.1 | science | side left | 0.1 | 1e-05 | 1 | 124 | 1024 | 0.025 → 0.000 |
science_left-kl1 | science | side left | 1.0 | 1e-05 | 1 | 124 | 1024 | 0.025 → 0.000 |
science_left-kl10 | science | side left | 10.0 | 1e-05 | 1 | 124 | 1024 | 0.024 → 0.010 |
science_random-kl0 | science | random | 0.0 | 1e-05 | 1 | 124 | 1024 | 0.894 → 0.232 |
science_random-kl0.1 | science | random | 0.1 | 1e-05 | 1 | 124 | 1024 | 0.896 → 0.232 |
science_random-kl1 | science | random | 1.0 | 1e-05 | 1 | 124 | 1024 | 0.897 → 0.235 |
science_random-kl10 | science | random | 10.0 | 1e-05 | 1 | 124 | 1024 | 0.896 → 0.233 |
science_right-kl0 | science | side right | 0.0 | 1e-05 | 1 | 124 | 1024 | 1.953 → 0.000 |
science_right-kl0.1 | science | side right | 0.1 | 1e-05 | 1 | 124 | 1024 | 1.949 → 0.000 |
science_right-kl1 | science | side right | 1.0 | 1e-05 | 1 | 124 | 1024 | 1.953 → 0.000 |
science_right-kl10 | science | side right | 10.0 | 1e-05 | 1 | 124 | 1024 | 1.950 → 0.009 |