Downloads · 30 days
9
14% of all-time downloads
bge-m3-ir-research-lora-v1
Lzvick/bge-m3-ir-research-lora-v1
bge-m3-ir-research-lora-v1 is a feature extraction model from Lzvick. Use it when you need embeddings to search or compare text. It is set up for FlagEmbedding. The card lists the license as mit.
LoRA fine-tune of BAAI/bge-m3 for dense retrieval over information-retrieval research literature (BM25, DPR, ColBERT, RAG, MS MARCO, BEIR). Weights below are the merged model (base + adapter) — no peft needed to load it.
All-time downloads
63
Public
Parameters
568M
2.3 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors2.3 GB · 99%
At a glance
- Task
- Feature Extraction
- Library
- FlagEmbedding
- License
- mit
- Model type
- xlm-roberta
- Access
- Public
- Created
- Jul 3, 2026
- Updated
- Jul 3, 2026
- SHA
- 24bb8a6e
Base models
Model card
From the Hugging Face model README
bge-m3-ir-research-lora-v1
LoRA fine-tune of BAAI/bge-m3 for dense retrieval over information-retrieval research literature (BM25, DPR, ColBERT, RAG, MS MARCO, BEIR). Weights below are the merged model (base + adapter) — no peft needed to load it.
Usage
from FlagEmbedding import BGEM3FlagModel
model = BGEM3FlagModel("Lzvick/bge-m3-ir-research-lora-v1", use_fp16=True)
output = model.encode(
["What is dense passage retrieval?"],
return_dense=True,
return_sparse=True,
return_colbert_vecs=False,
)
Training
| Setting | Value |
|---|---|
| Base model | BAAI/bge-m3 |
| Hardware | Single consumer GPU, 12GB VRAM |
| Method | LoRA (r=16, alpha=32, dropout=0.1, targets: query,key,value) |
| Trainable params | 2,359,296 / 570,114,048 (0.41%) |
| Epochs | 3 |
| Effective batch size | 128 (4 per-device × 32 grad-accum) |
| Learning rate | 1e-5 |
| Query / passage max len | 256 / 256 |
| Train group size | 8 (1 positive + 7 negatives per query) |
| Precision | bf16 |
| Train set | 2,072 examples |
Evaluation
Sanity check on 109 held-out validation examples: mean cosine-similarity margin between query→positive and query→negative pairs.
| Model | Mean pos-neg margin |
|---|---|
Baseline (BAAI/bge-m3) | 0.0849 |
| Fine-tuned (this model) | 0.0861 |
This is a coarse sanity check, not a retrieval benchmark (no nDCG/Recall/MRR test-set evaluation has been run yet). Treat this as an early checkpoint from a short run (~51 optimizer steps), not a fully validated release.
- Task
- Feature Extraction
- Library
- FlagEmbedding
- Type
- xlm-roberta
- License
- mit
- Created
- Jul 3, 2026
- Updated
- Jul 3, 2026