Downloads · 30 days
5
9% of all-time downloads
Seraphic663/lrat-qwen3-0.6b-2epoch-20260717
lrat-qwen3-0.6b-2epoch-20260717 is a feature extraction model from Seraphic663. Use it when you need embeddings to search or compare text. It is set up for transformers.
This is a full-parameter retriever checkpoint based on Qwen/Qwen3-Embedding-0.6B, prepared for the CCIR LRAT competition.
Downloads · 30 days
5
9% of all-time downloads
All-time downloads
57
Public
Parameters
596M
2.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors2.4 GB · 99%
From the Hugging Face model README
This is a full-parameter retriever checkpoint based on
Qwen/Qwen3-Embedding-0.6B, prepared for the CCIR LRAT competition.
Given a web search query, retrieve relevant passages that answer the queryThe second epoch starts from the first epoch's full model weights and uses a
freshly initialized optimizer and learning-rate schedule. Training uses the
official LRAT pair data and reweight_rate values with cross-device negatives.
The repository contains complete model parameters and does not require a LoRA
or adapter at inference time.
import torch
import torch.nn.functional as F
from transformers import AutoModel, AutoTokenizer
model_id = "Seraphic663/lrat-qwen3-0.6b-2epoch-20260717"
tokenizer = AutoTokenizer.from_pretrained(model_id, padding_side="left")
model = AutoModel.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
).cuda().eval()
instruction = "Given a web search query, retrieve relevant passages that answer the query"
texts = [f"Instruct: {instruction}\nQuery:example query", "example passage"]
batch = tokenizer(
texts,
padding=True,
truncation=True,
max_length=512,
return_tensors="pt",
).to("cuda")
with torch.inference_mode():
output = model(**batch).last_hidden_state
embeddings = F.normalize(output[:, -1].float(), p=2, dim=1)
On a fixed 500-row candidate-set training-overlap diagnostic, this checkpoint obtained Recall@1/5/10 of 0.738/0.942/0.990 and MRR of 0.82813. All 500 queries also occur in the training pairs, so these values are regression diagnostics, not held-out results or official BC-Plus leaderboard scores.
model.safetensors
2,383,139,480 bytes
SHA-256 1075292002c27eee654f83e1d1d568a78d4cc9238bc48a36d4ceb5f35968e050