AlignSurvey-Llama-3.1-8B

AlignSurvey Llama-3.1-8B Stage-I LoRA

This repository contains the Stage-I LoRA adapter for meta-llama/Llama-3.1-8B-Instruct. It was trained for one epoch on the AlignSurvey Social Foundation Corpus before any task-specific Stage-II fine-tuning.

Important base-model information

The original training run loaded the base model from <local-model-root>/Meta-Llama-3.1-8B-Instruct. The exact Hugging Face commit was not recorded in the training artifacts and must be confirmed from that server-side base-model directory before release. Do not claim a commit until it has been verified.

Tokenizer: use the tokenizer from the same base-model revision.

LLaMA-Factory template: llama3.

Access to the base model requires acceptance of Meta's Llama 3.1 license on Hugging Face.

LoRA configuration

• PEFT type: LoRA
• Rank: 8
• Alpha: 16
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj

Stage-I training

• Learning rate: 1e-4
• Scheduler: cosine
• Warmup ratio: 0.1
• Seed: 42

Loading

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "meta-llama/Llama-3.1-8B-Instruct"
adapter_dir = "path/to/adapter"

model = AutoModelForCausalLM.from_pretrained(base_model, torch_dtype="auto")
model = PeftModel.from_pretrained(model, adapter_dir)
tokenizer = AutoTokenizer.from_pretrained(base_model)

To create a merged model, call model.merge_and_unload() and then save_pretrained(). Distribution and use remain subject to the Meta Llama 3.1, dataset, and original survey-source licenses.