adapter-checkpoint

Adapter Checkpoint — LoRA on Llama-2-7b

This repository contains a LoRA adapter checkpoint fine-tuned on top of meta-llama/Llama-2-7b-hf using PEFT.

---

Repository layout

.
├── adapter_config.json          # PEFT / LoRA hyper-parameters
├── adapter_model.bin            # Trained adapter weights
├── README.md                    # This file
└── examples/
    └── chat/
        ├── zero_shot/
        │   └── prompt.json      # Zero-shot chat prompt template
        └── few_shot/
            └── prompt.json      # Few-shot chat prompt template

---

Prompt templates

Two ready-to-use prompt templates are included for chat inference:

---

Quick start

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
import json, pathlib

# Load adapter config and base model
config = PeftConfig.from_pretrained("dongbobo/adapter-checkpoint")
base   = AutoModelForCausalLM.from_pretrained(config.base_model_name_or_path)
model  = PeftModel.from_pretrained(base, "dongbobo/adapter-checkpoint")
tok    = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

# Load a prompt template
template = json.loads(
    pathlib.Path("examples/chat/zero_shot/prompt.json").read_text()
)

# Build prompt
user_msg = "Explain the concept of attention in transformers."
prompt   = (
    f"<s>[INST] <<SYS>>\n{template['template']['system']}\n<</SYS>>\n\n"
    f"{user_msg} [/INST]"
)

inputs  = tok(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tok.decode(outputs[0], skip_special_tokens=True))

---

Adapter hyper-parameters

---

License

Released under the Apache 2.0 license.
The base model (meta-llama/Llama-2-7b-hf) is subject to its own Llama 2 Community License.