Downloads · 30 days
0
just1nseo/olmo3-7b-think-if-rlvr-constraint-only-pp0-8k
olmo3-7b-think-if-rlvr-constraint-only-pp0-8k is a text generation model from just1nseo. Use it when you need the model to write or continue text. It is set up for transformers.
BF16 inference checkpoints for allenai/Olmo-3-7B-Think-DPO, trained with GRPO using only the instruction-following constraint reward. Anchor/PPL rewards are disabled.
Downloads · 30 days
0
Access
Public
Updated Aug 5, 2026
Repo size
—
Likes
0
Public
Click a slice to open those files.
.md2.1 KB · 58%
From the Hugging Face model README
BF16 inference checkpoints for allenai/Olmo-3-7B-Think-DPO, trained with GRPO using
only the instruction-following constraint reward. Anchor/PPL rewards are disabled.
Each checkpoint is a complete Transformers model in its own subfolder:
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "just1nseo/olmo3-7b-think-if-rlvr-constraint-only-pp0-8k"
subfolder = "global_step_364"
tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=subfolder)
model = AutoModelForCausalLM.from_pretrained(
repo,
subfolder=subfolder,
torch_dtype=torch.bfloat16,
device_map="auto",
)
OLMo 3 requires a Transformers version with native olmo3 support (the training runtime
used Transformers 4.57.1).
| Subfolder | Completed epoch |
|---|---|
global_step_91 | 1 |
global_step_182 | 2 |
global_step_273 | 3 |
global_step_364 | 4 |
| Setting | Value |
|---|---|
| Experiment | olmo3_7b_grpo_think_constraint_only_b1024_c1_pp0_8k |
| Base and reference model | allenai/Olmo-3-7B-Think-DPO |
| Training epochs / total steps | 4 / 364 |
| Train batch size | 1024 |
| Rollouts per prompt | 8 |
| Maximum prompt / response length | 2048 / 8192 |
| Rollout sampling | temperature 0.6, top-p 0.95, presence penalty 0.0 |
| Actor learning rate | 1e-6 |
| KL loss | low-variance KL, coefficient 0.001 |
| IF reward | fraction of satisfied IFEval constraints |
| Auxiliary PPL / anchor rewards | Disabled (both coefficients 0.0; anchor precompute/policy scoring off) |
| Reasoning format | OLMo Think template; a closing </think> is required for IF reward |
The full FSDP trainer checkpoints, including optimizer and data-loader state, are retained separately. This repository contains merged BF16 model weights intended for inference and evaluation.