dcl-llama2-7b-lr2e-5-lam0.1-eps1

DCL LoRA adapters (Llama-2-7B-Chat)

Low-rank adapters from Directed Consistency Learning fine-tuning of meta-llama/Llama-2-7b-chat-hf.

Released checkpoints

Training details

• Method: Directed consistency learning (asymmetric hinge on frozen Llama Guard critic + soft rollout on perturbed prompt).
• LoRA: rank 8, α=16, dropout 0.05, targets q_proj, v_proj.
• Optimizer: AdamW, lr 2e-5, 5 epochs, batch size 1.
• Data: 3,000 harmful prompts with GCG / AutoDAN / PAIR neighbours (see data/MaliciousInstruct++/ for the MaliciousInstruct++ portion).

How to load

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "meta-llama/Llama-2-7b-chat-hf"
adapter = "anon7302/dcl-llama2-7b-lr2e-5-lam0.1-eps1"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

Train a new adapter from this repository:

python train.py \
  --model-profile llama_2_7b_chat \
  --training-data data/train/train_plus_validation.csv \
  --finetuned-llm-path ./checkpoints/dcl \
  --lr 2e-5 \
  --lambda-val 0.1 \
  --epsilon 1.0 \
  --lm-loss-input clean \
  --total-epochs 5 \
  --start-epoch 1

Evaluation

ASR / FRR protocol matches HarmBench + JailbreakBench (paper §4):

python eval/eval.py \
  --model-profile llama_2_7b_chat \
  --resume-from anon7302/dcl-llama2-7b-lr2e-5-lam0.1-eps1 \
  --validation-data data/eval/harmful_validation.csv \
  --benign-validation-data data/eval/frr_validation.csv \
  --eval-mode seen-family

Limitations

• Not safe for deployment; jailbreaks remain possible.
• Tuned on Llama-2-7B-Chat only; transfer to other bases is not guaranteed.
• Harmful fine-tuning context — use only in controlled research settings.

License

Adapter weights inherit Llama 2 terms from the base model. Repository code is MIT.