Downloads · 30 days
32
100% of all-time downloads
olmo2-7b-instruct-inv-reference-mixed-klrev
MARS-Retokenization/olmo2-7b-instruct-inv-reference-mixed-klrev
olmo2-7b-instruct-inv-reference-mixed-klrev is a machine learning model from MARS-Retokenization. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Research checkpoint from a study of tokenization (reader) invariance and its effect on robustness to adversarial re-tokenization. Fine-tuned from allenai/OLMo-2-1124-7B-Instruct.
All-time downloads
32
Public
Parameters
7.3B
14.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.6 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- olmo2
- Access
- Public
- Created
- Sep 8, 2026
- Updated
- Sep 8, 2026
- SHA
- 3a0f7dbc
Base models
Model card
From the Hugging Face model README
inv_reference_mixed_klrev
Research checkpoint from a study of tokenization (reader) invariance and its
effect on robustness to adversarial re-tokenization. Fine-tuned from
allenai/OLMo-2-1124-7B-Instruct.
These are research artifacts, not products. Numbers below are measured on a 200-prompt AdvBench holdout that was excluded from training by construction.
Measured
| metric | value |
|---|---|
| — | not yet measured |
AdvTok ASR is attack success rate under adversarial tokenization
(Geh et al., arXiv:2503.02174), Llama-Guard-3-8B judged, greedy decoding.
XSTest over-refusal is the refusal rate on safe prompts — the cost side.
Training configuration
| field | value |
|---|---|
mode | reference |
objective | invariance |
kl_direction | reverse |
ce_weighting | uniform |
ema_beta | None |
harmful_mix | mixed |
harmful_fraction | 0.286 |
num_encodings | 8 |
cvar_quantile | 0.25 |
max_steps | 700 |
learning_rate | 1e-05 |
grad_accum | 8 |
seed | 42 |
reference_model | allenai/OLMo-2-1124-7B-Instruct |
max_new_tokens | 128 |
prefix_tokens | 8 |
Parameter drift from base (training-happened guard)
| group | relative L2 |
|---|---|
| attn | 0.01202 |
| embed_tokens | 0.00096 |
| lm_head | 0.00605 |
| mlp | 0.01320 |
| norm | 0.00101 |
Caveats
- Single seed. No claim of significance across seeds.
- Evaluated on English AdvBench/XSTest/Alpaca only.
- The safety numbers are for the specific attack studied; they do not imply robustness to other jailbreaks.
- Type
- olmo2
- License
- apache-2.0
- Created
- Sep 8, 2026
- Updated
- Sep 8, 2026