thesis-loras-maiya-baseline

Maiya et al. (2025) OpenCharacterTraining baseline -- full re-run

Our reproduction of the OpenCharacterTraining pipeline (\citet{maiya2025opencharactertrainingshaping}), used as the primary baseline condition throughout Thesis Chapter 4 (H1, H2, H3), for Llama-3.1-8B-Instruct and OLMo-3-7B-Instruct.

Pipeline stages, each a LoRA trained on top of the previous stage:

1. {llama,Olmo}-distillation/<persona> -- constitution distillation SFT
2. {llama,Olmo}-introspection/<persona> -- self-reflection/self-interaction SFT
3. {llama,Olmo}-personas/<persona> -- final DPO-tuned persona LoRA (this is the checkpoint evaluated in the thesis tables)

Personas: goodness, misalignment, sarcasm.

Base models (merge/load the LoRA on top of these):

• llama-* / Olmo-* (capital O) adapters: meta-llama/Llama-3.1-8B-Instruct
• olmo-* (lowercase o) adapters: allenai/Olmo-3-7B-Instruct-SFT

Training data for each stage: BeschB/thesis-training-data (maiya/{dpo,distillation,preferences,self_interaction,self_reflection,sft_data}).

Note: Maiya et al.'s own originally-released checkpoints (used only to validate our re-run against theirs, not as a trained artifact of this project) are not re-hosted here -- see their release at maius/llama-3.1-8b-it-personas on HuggingFace.