vit_large_patch14_224_skyclip_30pct

Model rehosted from https://github.com/wangzhecheng/SkyScript

Original source: https://opendatasharing.s3.us-west-2.amazonaws.com/SkyScript/ckpt/SkyCLIP_ViT_L14_top30pct.zip

Model weights extracted below:

import os
import torch
import hashlib

import timm
from open_clip.factory import create_model_and_transforms
from timm.models.vision_transformer import _convert_openai_clip


path = "SkyCLIP_ViT_L14_top30pct/epoch_20.pt"
encoder = "ViT-L-14"
model, _, preprocess_val = create_model_and_transforms(encoder, path, weights_only=False)
print(preprocess_val)
model_timm = timm.create_model("vit_large_patch14_clip_224", pretrained=False, num_classes=768)
converted = _convert_openai_clip(model.state_dict(), model_timm)
model_timm.load_state_dict(converted, strict=True)

filename = "vit_large_patch14_224_skyclip_30pct.pth"
torch.save(model_timm.state_dict(), filename)
md5 = hashlib.md5(open(filename, "rb").read()).hexdigest()[:8]
os.rename(filename, filename.replace(".pth", f"-{md5}.pth"))

The preprocessing transforms are:

Compose(
    Resize(size=224, interpolation=bicubic, max_size=None, antialias=True)
    CenterCrop(size=(224, 224))
    <function _convert_to_rgb at 0x33047ff60>
    ToTensor()
    Normalize(mean=(0.48145466, 0.4578275, 0.40821073), std=(0.26862954, 0.26130258, 0.27577711))
)