siglip2-large-lora-v1

SigLIP2-Large LoRA v1

LoRA fine-tune of google/siglip2-large-patch16-384 for material/surface visual-similarity retrieval. Trained with Supervised Contrastive loss on catalog↔in-room segment pairs.

Best validation metrics

Training configuration

• Base model: google/siglip2-large-patch16-384 (vision encoder)
• Adapter: LoRA, α=32, r=16, dropout=0.1
• Target modules: q_proj, k_proj, v_proj, out_proj (attention)
• Loss: Supervised Contrastive (SupCon)
• Temperature: 0.07
• Batch size: 32 (×1 grad accum)
• Images per class: 2
• Optimizer: AdamW, lr=1e-4, weight_decay=0.01
• Schedule: cosine, 10% warmup
• Epochs trained: 4
• Final train loss: 0.2828

Per-epoch metrics

• intra — mean cosine similarity for same-class pairs
• inter — mean cosine similarity for cross-class pairs
• sep — intra / inter (higher = better class separation)

Companion dataset

subhrokomol/siglip2-large-lora-v1-dataset

Load

from peft import PeftModel
from transformers import AutoModel, AutoProcessor

base = AutoModel.from_pretrained("google/siglip2-large-patch16-384")
processor = AutoProcessor.from_pretrained("google/siglip2-large-patch16-384")
model = PeftModel.from_pretrained(base, "subhrokomol/siglip2-large-lora-v1")
model.eval()

# image embedding (1024-dim)
import torch
from PIL import Image
img = Image.open("your_image.jpg").convert("RGB")
inputs = processor(images=img, return_tensors="pt")
with torch.no_grad():
    embed = model.get_image_features(**inputs)
embed = embed / embed.norm(dim=-1, keepdim=True)  # L2-normalize

Intended use

Visual-similarity retrieval over materials, surfaces, and textures — e.g. given a crop from an interior photo, retrieve catalog products with similar visual appearance. Cross-domain (in-the-wild photo ↔ catalog studio photo) is the primary use case.

License

Apache 2.0.