memory-retrieval-jina-v3-lora-merged

memory-retrieval-jina-v3-lora-merged

🧠 Fine-tuned Jina Embeddings v3 Model (Merged LoRA Weights)

This repository contains a merged full model of jinaai/jina-embeddings-v3, fine-tuned via LoRA rank 24 for memory retrieval tasks such as:

• long-term personal memory storage
• chatbot memory recall
• RAG-based agent memory
• document → fact extraction
• high-recall semantic search

The LoRA weights have been merged into the base model, producing a full standalone embedding model (~2.38GB) compatible with transformers and sentence-transformers.

---

🔧 Training Overview

• Base Model: jinaai/jina-embeddings-v3
• Fine-tuning Method: LoRA (r=24)
• Optimizer: AdamW
• Loss Function: Triplet Loss (semi-hard) + Cosine Similarity
• Batch Size: 256
• Max Seq Length: 640
• Epochs: 1
• Framework: SentenceTransformers + PEFT

The model was optimized specifically for improving retrieval of personalized chat memories, including:

• short-term and long-term memories
• user preferences
• user profile data
• conversation summaries
• factual statements about the user

---

📚 Training Data

The model was trained on a custom synthetic dataset generated using a structured schema:

• Memory statements
• Queries
• Positive passages
• Hard negative passages

Each sample followed:

{
  "query": "...",
  "positive": "...",
  "negative": "..."
}

🚀 Usage with SentenceTransformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "Mercity/memory-retrieval-jina-v3-lora-merged",
    trust_remote_code=True
)

emb = model.encode("What did I tell you about my dog?")
print(emb.shape)