AdvFLYP

AdvFLYP

Official model checkpoints for Finetune Like You Pretrain: Boosting Zero-shot Adversarial Robustness in Vision-language Models (CVPR Findings 2026).

• Paper: arXiv:2604.11576
• Code and evaluation instructions: Sxing2/AdvFLYP

AdvFLYP adversarially finetunes CLIP ViT-B/32 on one million image-text pairs sampled from LAION-400M. The full variant adds logit- and feature-level regularisation.

Checkpoints

The files are the original PyTorch training checkpoints released by the authors. They contain the finetuned partial state dict and training state expected by the official repository.

Download

from huggingface_hub import hf_hub_download

checkpoint_path = hf_hub_download(
    repo_id="xingslong/AdvFLYP",
    filename="AdvFLYP_full_checkpoint.pth.tar",
)
print(checkpoint_path)

For the non-regularised checkpoint, set filename="AdvFLYP_NonReg_checkpoint.pth.tar".

Evaluation

Clone the official repository and prepare its environment and datasets as described there. Then run:

git clone https://github.com/Sxing2/AdvFLYP.git
cd AdvFLYP

TEST_MODEL_PATH=/path/to/AdvFLYP_full_checkpoint.pth.tar
TEST_SET=(cifar10 cifar100 STL10 Caltech101 Caltech256 oxfordpet flowers102 Food101 StanfordCars SUN397 Country211 fgvc_aircraft EuroSAT dtd)

python -m code.main \
  --evaluate \
  --root ./data \
  --resume "$TEST_MODEL_PATH" \
  --test_set "${TEST_SET[@]}" \
  --batch_size 256 \
  --test_attack_type pgd \
  --test_eps 1 \
  --test_numsteps 10 \
  --test_stepsize 1

The official implementation initializes OpenAI CLIP ViT-B/32 and loads the checkpoint's partial_state_dict with strict=False.

Training data

The models were finetuned on one million reachable image-text pairs randomly sampled from LAION-400M. See the GitHub repository for the released data and preparation details.

Intended use and limitations

These checkpoints are intended for research on zero-shot image classification and adversarial robustness. Evaluation in the paper covers 14 downstream datasets under the stated attack configuration. Performance outside those datasets, threat models, and preprocessing settings has not been established.

Because the training images originate from web-scale LAION data, the checkpoints may inherit biases and other limitations of CLIP and the underlying data.

Citation

@InProceedings{Xing_2026_CVPR,
  author    = {Xing, Songlong and Wang, Weijie and Zhao, Zhengyu and Gu, Jindong and Torr, Philip and Sebe, Nicu},
  title     = {Finetune Like You Pretrain: Boosting Zero-shot Adversarial Robustness in Vision-language Models},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings},
  month     = {June},
  year      = {2026},
  pages     = {737--747}
}