OceanCLIP-0.15B

zjunlp/OceanCLIP-0.15B

OceanCLIP-0.15B is a zero-shot classification model from zjunlp. Use it when you need labels you did not train the model on. It is set up for open_clip. The card lists the license as apache-2.0.

A vision-language model fine-tuned on marine imagery and biological terminology using the OpenCLIP framework. Built upon BioCLIP, it is optimized for marine species identification, zero-shot classification, and cross-…

Downloads · 30 days

Access

Public

Updated May 14, 2026

Repo size

2.4 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.pt1.8 GB · 75%

At a glance

Task: Zero-Shot Classification
Library: open_clip
License: apache-2.0
Access: Public
Created: May 10, 2026
Updated: May 14, 2026
SHA: bdfbcec0

Try a prompt

Model card

From the Hugging Face model README

OceanCLIP-0.15B: Marine Vision-Language Model

📂 Repository Contents

Directory	File	Description
`oceanclip-bio/`	`epoch_50.pt`	Fine-tuned checkpoint. Marine-adapted weights after 50 training epochs. Contains the updated vision & text encoder projections.
`oceanclip-bio/`	`terms.txt`	Marine terminology list. Line-by-line species names (e.g., `A abramis`). Used for zero-shot classification to dynamically build class-specific text prompts.
`bioclip/`	`open_clip_config.json`	Architecture & preprocessing config. Defines ViT-B/16 vision encoder, Transformer text encoder (77 context, 512 width), and image normalization (`mean`/`std`).
`bioclip/`	`open_clip_pytorch_model.bin`	Base BioCLIP weights. Original OpenCLIP-format pre-trained weights. Serves as the initialization backbone before marine-specific fine-tuning.

🚀 Usage

Requires open_clip_torch and torch.

import open_clip
import torch
from PIL import Image

# 1. Load architecture & base weights
model, _, preprocess = open_clip.create_model_and_transforms(
    model_name="ViT-B-16",
    pretrained="bioclip/open_clip_pytorch_model.bin"
)
tokenizer = open_clip.get_tokenizer("ViT-B-16")

# 2. Load fine-tuned marine weights
state_dict = torch.load("oceanclip-bio/epoch_50.pt", map_location="cpu")
model.load_state_dict(state_dict, strict=False)
model.eval()

# 3. Inference (Zero-Shot with terms.txt)
image = preprocess(Image.open("marine_input.jpg")).unsqueeze(0)
terms = [line.strip() for line in open("oceanclip-bio/terms.txt", "r") if line.strip()]
text_tokens = tokenizer(terms)

with torch.no_grad():
    image_feat = model.encode_image(image)
    text_feat = model.encode_text(text_tokens)
    logits = (image_feat @ text_feat.T).softmax(dim=-1)
    
    top_idx = logits.argmax().item()
    print(f"Predicted species: {terms[top_idx]} (Confidence: {logits[0, top_idx]:.4f})")

Task: Zero-Shot Classification
Library: open_clip
License: apache-2.0
Languages: en
Created: May 10, 2026
Updated: May 14, 2026