Downloads · 30 days
6
32% of all-time downloads
opd-exopd-lambda1.0-klref-step5
ntlfi/opd-exopd-lambda1.0-klref-step5
opd-exopd-lambda1.0-klref-step5 is a machine learning model from ntlfi. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.
Intermediate checkpoint from a smoke test of a fix for a reasoning-shortcut reward hack found in On-Policy Distillation with Reward Extrapolation (ExOPD) training. Full context: ntlfi/opdtraining (see SESSIONLOG.md an…
All-time downloads
19
Public
Parameters
8B
16.1 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors16.1 GB · 100%
At a glance
- License
- llama3
- Model type
- llama
- Access
- Public
- Created
- Aug 11, 2026
- Updated
- Aug 11, 2026
- SHA
- 545512ec
Base models
Model card
From the Hugging Face model README
ExOPD λ=1.0 + KL-to-reference trust region, step 5/15
Intermediate checkpoint from a smoke test of a fix for a reasoning-shortcut
reward hack found in On-Policy Distillation with Reward Extrapolation
(ExOPD) training. Full context:
ntlfi/opd_training (see
SESSION_LOG.md and algos/exopd_incident_2026-08-07.md).
Not formally evaluated (only step10 and step15/final were run through the full GSM8K test set). Training-time metrics at this step: loss=0.192, kl_penalty=0.0003 (very small — most of the drift measured by the KL term happens later in the run, see step10/step15 for comparison).
Teacher: meta-llama/Meta-Llama-3-70B-Instruct (frozen). Student/base:
meta-llama/Meta-Llama-3-8B-Instruct. λ=1.0 (standard on-policy distillation
reduction, no reward-extrapolation reference term active), plus an added
KL(π_θ‖π_ref) trust-region penalty (kl_ref_coef=0.1, not part of the
published method — added to counter a reward-hacking failure mode this
repo found independently) using Schulman's k3 estimator.
- Type
- llama
- License
- llama3
- Created
- Aug 11, 2026
- Updated
- Aug 11, 2026