self-align-to-explain-adapters

Self-Align to Explain — Best Adapters

The best LoRA adapter for each model × dataset × method cell (72 in total) from the thesis Self-Align to Explain: Comparing Post-Training Methods for Counterfactual Generation. Each adapter fine-tunes an instruction-tuned base model to generate minimal, label-flipping counterfactuals for BoolQ, SNLI-Premise, or SNLI-Hypothesis, using one of SFT, DPO, SimPO, GRPO (composite or decomposed reward), or GDPO.

Code, exact per-run training configurations, and full results: https://github.com/fledor/self-align-to-explain (see reports/BEST_CONFIGS.md).

Contents

Each subfolder <base>_<dataset>_<method> contains the LoRA adapter (adapter_config.json, adapter_model.safetensors), the tokenizer and chat template used, training_args.bin, the run's wandb_config.yaml and wandb_summary.json (training provenance), and METRICS.json (evaluation stats). All adapters are QLoRA rank 32, α=16, over all attention + MLP projections.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", torch_dtype="bfloat16")
model = PeftModel.from_pretrained(base, "fledor/self-align-to-explain-adapters",
                                  subfolder="qwen7b_snli_premise_simpo")

Licensing

The adapter weights are released under MIT where the base model license permits; adapters of meta-llama/Llama-3.1-8B-Instruct are additionally subject to the Llama 3.1 Community License, and adapters of Qwen/Qwen2.5-3B-Instruct to the Qwen Research License, as derivatives of those base models.

Evaluation (fair frozen-base, N=200 prompts x 10 counterfactuals)

LFR = label flip rate (judged by the unmodified base model, against its own prediction on the original input). ΔLFR in percentage points vs the frozen base anchor of the cell. NED = median normalized edit distance over non-trivial edits; PPL = median perplexity; Parse% = unique parsed counterfactuals / attempts.