Qwen3-8b-lora-narrativeqa

Qwen3-8b-lora-narrativeqa

LoRA adapter for Qwen/Qwen3-8B fine-tuned on the narrativeqa RAG generative dataset (DinoStackAI/narrativeqa-rag).

• Best dev metric: eval_loss = 0.9738

Load with PEFT

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base_model, "DinoStackAI/Qwen3-8b-lora-narrativeqa")
tokenizer = AutoTokenizer.from_pretrained("DinoStackAI/Qwen3-8b-lora-narrativeqa")

Load with vLLM (LoRA)

from vllm import LLM
from vllm.lora.request import LoRARequest

llm = LLM(
    model="Qwen/Qwen3-8B",
    enable_lora=True,
    max_lora_rank=16,
)
outputs = llm.generate(
    prompts,
    lora_request=LoRARequest("narrativeqa", 1, "DinoStackAI/Qwen3-8b-lora-narrativeqa"),
)

Use this adapter with scripts/generation/run_rag_generation.py --lora-path DinoStackAI/Qwen3-8b-lora-narrativeqa.

Training details

• Base model: Qwen/Qwen3-8B
• Fine-tuning dataset: DinoStackAI/narrativeqa-rag
• Method: LoRA (r=16, lora_alpha=32, lora_dropout=0.05)
• Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• Loss: SFT with completion-only masking (assistant_only_loss=True)
• Best checkpoint selection: dev eval_loss