Llama-3-8B-D-STEER-OpenHermes-IT

D-STEER · Llama-3-8B OpenHermes — IT checkpoint

This repository is the canonical D-STEER release of the Llama-3-8B OpenHermes instruction-tuned checkpoint.

D-STEER is a research programme on activation steering for safety behaviour in language models: whether a single direction in activation space, estimated from the difference between a preference-tuned checkpoint and its instruction-tuned starting point, can install or remove harmlessness behaviour at inference time.

Within D-STEER this IT checkpoint is the starting point of the model pair: the reference against which harmlessness is installed, and the endpoint that steering is applied to when installing safety behaviour.

Model details

• Base foundation model: meta-llama/Meta-Llama-3-8B, revision 8cde5ca8380496c9a6cc7ef3a8b46a0372a1d920
• Instruction-tuning data: OpenHermes-2.5
• Format: self-contained merged causal language model (no adapter required)
• Stored precision: float16 (all 291 tensors)
• Architecture: Llama, 32 layers, hidden size 4096, vocabulary 128,257

Instruction tuning was implemented through supervised fine-tuning on OpenHermes-2.5. The released weights are fully merged, so the repository loads directly with AutoModelForCausalLM and needs no PEFT adapter.

Requires transformers >= 5.0. The tokenizer in this repository is stored in the current tokenizers-backend format. Loading it with transformers 4.x raises Tokenizer class TokenizersBackend does not exist.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "samarthraina/Llama-3-8B-D-STEER-OpenHermes-IT"

tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.float16)
model.eval()

messages = [{"role": "user", "content": "Explain activation steering in two sentences."}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
)
out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Intended use

Released for AI-safety research: measuring and comparing alignment behaviour, and serving as the unsteered reference endpoint in activation-steering experiments.

Limitations and safety

• This is an instruction-tuned research checkpoint with no additional harmlessness training. It is the less safe member of its model pair by construction, and it will comply with harmful requests more often than the preference-tuned checkpoint.
• Not evaluated or hardened for production deployment. Do not use it as a safety model.
• Inherits the limitations, biases, and knowledge cutoff of Meta Llama 3 8B and of the OpenHermes-2.5 instruction data.
• English-centric; behaviour in other languages is untested here.

Provenance and verification

PROVENANCE.json in this repository records the SHA-256 of every published file.

Matched checkpoint

The preference-tuned counterpart is samarthraina/Llama-3-8B-D-STEER-OpenHermes-DPO.

License and attribution

Built with Meta Llama 3.

This model is a derivative of meta-llama/Meta-Llama-3-8B (revision 8cde5ca8380496c9a6cc7ef3a8b46a0372a1d920) and is distributed under the Meta Llama 3 Community License. Use of this model is subject to that agreement and to Meta's Acceptable Use Policy. By using these weights you agree to both.

• Meta Llama 3 Community License: https://llama.meta.com/llama3/license/
• Meta Llama 3 Acceptable Use Policy: https://llama.meta.com/llama3/use-policy/
• Base foundation model: meta-llama/Meta-Llama-3-8B

This repository does not contain a foundation model trained from scratch. It is a research checkpoint derived from Meta Llama 3 8B.