Downloads · 30 days
12
30% of all-time downloads
olmo2-7b-instruct-inv-prefix-mixed-self-sft
MARS-Retokenization/olmo2-7b-instruct-inv-prefix-mixed-self-sft
olmo2-7b-instruct-inv-prefix-mixed-self-sft is a machine learning model from MARS-Retokenization. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Research checkpoint from a study of tokenization (reader) invariance and its effect on robustness to adversarial re-tokenization. Fine-tuned from allenai/OLMo-2-1124-7B-Instruct.
All-time downloads
40
Public
Parameters
7.3B
14.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors14.6 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- olmo2
- Access
- Public
- Created
- Sep 8, 2026
- Updated
- Sep 8, 2026
- SHA
- 94c9fcb4
Base models
Model card
From the Hugging Face model README
inv_prefix_mixed_self_sft
Research checkpoint from a study of tokenization (reader) invariance and its
effect on robustness to adversarial re-tokenization. Fine-tuned from
allenai/OLMo-2-1124-7B-Instruct.
These are research artifacts, not products. Numbers below are measured on a 200-prompt AdvBench holdout that was excluded from training by construction.
Measured
| metric | value |
|---|---|
| AdvTok ASR (greedy) | 0.55 |
| AdvTok canonical ASR (greedy) | 0.115 |
| AdvTok ASR (t=1) | 0.554 |
| AdvTok canonical ASR (t=1) | 0.16 |
AdvTok ASR is attack success rate under adversarial tokenization
(Geh et al., arXiv:2503.02174), Llama-Guard-3-8B judged, greedy decoding.
XSTest over-refusal is the refusal rate on safe prompts — the cost side.
Training configuration
| field | value |
|---|---|
mode | prefix |
objective | self_sft |
harmful_mix | mixed |
harmful_fraction | 0.286 |
num_encodings | 8 |
cvar_quantile | 0.25 |
max_steps | 700 |
learning_rate | 1e-05 |
grad_accum | 8 |
seed | 42 |
reference_model | allenai/OLMo-2-1124-7B-Instruct |
max_new_tokens | 128 |
prefix_tokens | 8 |
Parameter drift from base (training-happened guard)
| group | relative L2 |
|---|---|
| attn | 0.01154 |
| embed_tokens | 0.00048 |
| lm_head | 0.00835 |
| mlp | 0.01240 |
| norm | 0.00097 |
Caveats
- Single seed. No claim of significance across seeds.
- Evaluated on English AdvBench/XSTest/Alpaca only.
- The safety numbers are for the specific attack studied; they do not imply robustness to other jailbreaks.
- Type
- olmo2
- License
- apache-2.0
- Created
- Sep 8, 2026
- Updated
- Sep 8, 2026