Downloads · 30 days
621
34% of all-time downloads
JianLiao/siglip2-spectrum-icons-naflex
siglip2-spectrum-icons-naflex is a feature extraction model from JianLiao. Use it when you need embeddings to search or compare text. It is set up for transformers.
This repository hosts a fine-tuned checkpoint derived from google/siglip2-base-patch16-naflex. The model keeps the SigLIP2 architecture and tokenizer from the base checkpoint and is optimized for: Image-text retrieval…
Downloads · 30 days
621
34% of all-time downloads
All-time downloads
1.8K
Public
Parameters
375M
1.5 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors1.5 GB · 100%
From the Hugging Face model README
This repository hosts a fine-tuned checkpoint derived from google/siglip2-base-patch16-naflex. The model keeps the SigLIP2 architecture and tokenizer from the base checkpoint and is optimized for: Image-text retrieval and caption alignment for Spectrum iconography.
import torch
from transformers import AutoModel, AutoProcessor
from PIL import Image
processor = AutoProcessor.from_pretrained("JianLiao/siglip2-spectrum-icons-naflex", use_fast=False)
model = AutoModel.from_pretrained("JianLiao/siglip2-spectrum-icons-naflex", dtype=torch.float16, attn_implementation="sdpa")
image = Image.open("./image.png").convert("RGB")
inputs = processor(
text=["display forecast", "Crystal ball with a small sparkle", "show prediction", "Minimalist fortune-telling orb on a stand", "Monochrome magic globe with star accent"],
images=[image],
return_tensors="pt",
padding="max_length",
max_num_patches=256
)
with torch.no_grad():
outputs = model(**inputs)
image_embeds = outputs.vision_model_output.pooler_output
text_embeds = outputs.text_model_output.pooler_output
image_embeds = image_embeds / image_embeds.norm(dim=-1, keepdim=True)
text_embeds = text_embeds / text_embeds.norm(dim=-1, keepdim=True)
similarity = text_embeds @ image_embeds.T
print(similarity)
CPU example output:
tensor([[0.1677],
[0.0732],
[0.1676],
[0.1084],
[0.1381]], dtype=torch.float16)
Captions 1/3 rank highest for the icon; caption 5 remains competitive without losing descriptiveness. Sample icon: