vit_base_patch32_224_skyclip_50pct

Model rehosted from https://github.com/wangzhecheng/SkyScript

Original source: https://opendatasharing.s3.us-west-2.amazonaws.com/SkyScript/ckpt/SkyCLIP_ViT_B32_top50pct.zip

Model weights extracted below:

import os
import torch
import hashlib

import timm
from open_clip.factory import create_model_and_transforms
from timm.models.vision_transformer import _convert_openai_clip


path = "checkpoints/SkyCLIP_ViT_B32_top50pct/epoch_20.pt"
encoder = "ViT-B-32"
model, _, preprocess_val = create_model_and_transforms(encoder, path, weights_only=False)
print(preprocess_val)
model_timm = timm.create_model("vit_base_patch32_clip_224", pretrained=False, num_classes=512)
converted = _convert_openai_clip(model.state_dict(), model_timm)
model_timm.load_state_dict(converted, strict=True)

filename = "vit_base_patch32_224_skyclip_50pct.pth"
torch.save(model_timm.state_dict(), filename)
md5 = hashlib.md5(open(filename, "rb").read()).hexdigest()[:8]
os.rename(filename, filename.replace(".pth", f"-{md5}.pth"))

The preprocessing transforms are:

Compose(
    Resize(size=224, interpolation=bicubic, max_size=None, antialias=True)
    CenterCrop(size=(224, 224))
    <function _convert_to_rgb at 0x77355e3f28e0>
    ToTensor()
    Normalize(mean=(0.48145466, 0.4578275, 0.40821073), std=(0.26862954, 0.26130258, 0.27577711))
)