siglip2_so400m_p16_512

See our collection for all versions of SigLIP 2.

Run SigLIP 2 with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/siglip2_so400m_p16_512

Paper: SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features (arXiv:2502.14786) · HF Papers

SigLIP 2 keeps SigLIP's sigmoid loss and adds captioning-based pretraining, self-distillation, and masked prediction for stronger dense features. It uses a 256k multilingual Gemma vocabulary, so many languages work without a separate multilingual checkpoint.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of google/siglip2-so400m-patch16-512 for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a zero-shot image-text checkpoint (SigLIP2ZeroShotClassify): pass image(s) and text prompts at inference time.

✨ Quick start

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

from zeromodels.models.siglip2 import (
    SigLIP2Processor,
    SigLIP2ZeroShotClassify,
)

processor = SigLIP2Processor.from_weights("zeromodels/siglip2_so400m_p16_512")
model = SigLIP2ZeroShotClassify.from_weights("zeromodels/siglip2_so400m_p16_512")

labels = [
    "a photo of a cat",
    "a photo of a dog",
    "a photo of a car",
    "a photo of a living room",
]
inputs = processor(text=labels, image_paths="your_image.jpg")
output = model(
    {
        "images": inputs["images"],
        "token_ids": inputs["input_ids"],
    }
)
print(output["image_logits"].shape)

Load any SigLIP 2 variant the same way with from_weights("zeromodels/<variant>"):

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• Prefer Processor.from_weights(...) so image size and tokenizer match the variant.
• Map processor input_ids to model token_ids. No padding mask is required.
• Tokenizer is Gemma-based (multilingual); prefer Processor.from_weights.
• See SigLIP 2 docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. SigLIP2ZeroShotClassify.from_weights("hf:google/siglip2-so400m-patch16-512").

Special Thanks

A huge thank you to the Google SigLIP 2 authors for creating and releasing these models.

License: Apache 2.0.