siglip

SigLIP: Optimized for AMD ROCm

SigLIP is a vision-language embedding model evaluated here for zero-shot image classification on CIFAR-10. This repository packages inference for zero-shot image classification using PyTorch (CPU/GPU) and vLLM (GPU pooling runner), exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs.

This is based on the implementation of SigLIP found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the siglip AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.

---

Task Overview

Task: Zero-shot image classification

Dataset: CIFAR-10 test split (10,000 images, 10 classes)

Output metrics: Accuracy (%), image throughput (img/s)

Model variants: Default is base-224 (google/siglip-base-patch16-224). Override with MODEL_VARIANT=base-256|base-384|large-256|large-384|so400m-224|so400m-384.

vLLM note: vLLM's SigLIP integration handles one modality per request (text batch, then one image per request); the client combines embeddings manually — the same approach as the upstream evaluation scripts.

---

AMD ROCm Optimization

This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs. Key points:

• Validated backends: PyTorch (native ROCm HIP kernels, CPU and GPU) and vLLM (ROCm-enabled, built from source, pooling runner on GPU).
• No code changes required versus the upstream SigLIP implementation — only environment/runtime configuration differs.
• CPU fallback path supported via PyTorch for environments without a ROCm-capable GPU.

---

Getting Started

For setup instructions, evaluation scripts, and custom configuration options, see the siglip on GitHub.

---

Model Details

Model Type: Vision-language embedding model (zero-shot image classification)

Base Model: google/siglip-base-patch16-224

Model Stats:

• Model variant: base-224 (default) — base-256 / base-384 / large-256 / large-384 / so400m-224 / so400m-384 also supported
• Precision tested: FP32, FP16, BF16 (CPU, GPU PyTorch, GPU vLLM)

---

Accuracy Pipeline

Higher accuracy means a larger fraction of images are assigned the correct CIFAR-10 label — 100% is perfect, 10% is random chance for this 10-way task. Values above ~90% on CIFAR-10 with SigLIP-base are typical; larger variants (large, so400m) often reach ~96–97%.

Metrics Explained

Accuracy Results

Full Dataset Evaluation (CIFAR-10 test) — siglip-base-patch16-224:

<!-- accuracy-table-start --> <!-- accuracy-table-end -->

---

Dig Deeper

Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?

📂 View the full project on GitHub

The GitHub repository includes:

• Setup and prerequisites for ROCm environments
• Scripts for both PyTorch and vLLM runners
• Additional model variants and datasets
• Benchmarking and reproduction instructions