Qwen3-Emb-4b-lora-narrativeqa

Qwen3-Emb-4b-lora-narrativeqa

LoRA adapter for Qwen/Qwen3-Embedding-4B fine-tuned on the narrativeqa RAG retrieval dataset (DinoStackAI/narrativeqa-rag).

• Best dev metric: eval_narrativeqa-dev_cosine_ndcg@10 = 0.8110

Load with Sentence Transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("DinoStackAI/Qwen3-Emb-4b-lora-narrativeqa")
embeddings = model.encode(["Instruct: ...\nQuery:your query", "document text"])

Or load the base model and adapter explicitly:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("Qwen/Qwen3-Embedding-4B")
model.load_adapter("DinoStackAI/Qwen3-Emb-4b-lora-narrativeqa")

Load with vLLM (LoRA)

from vllm import LLM
from vllm.lora.request import LoRARequest

llm = LLM(
    model="Qwen/Qwen3-Embedding-4B",
    task="embed",
    enable_lora=True,
    max_lora_rank=16,
)
outputs = llm.embed(
    ["Instruct: ...\nQuery:your query"],
    lora_request=LoRARequest("narrativeqa", 1, "DinoStackAI/Qwen3-Emb-4b-lora-narrativeqa"),
)

Training details

• Base model: Qwen/Qwen3-Embedding-4B
• Fine-tuning dataset: DinoStackAI/narrativeqa-rag
• Method: LoRA (r=16, lora_alpha=32, targets q_proj / v_proj)
• Loss: CachedMultipleNegativesRankingLoss
• Best checkpoint selection: dev IR NDCG@10