Downloads · 30 days
0
xn6o/lora-vit-large-patch16-224-in21k-r64-imagenet1k
lora-vit-large-patch16-224-in21k-r64-imagenet1k is a machine learning model from xn6o. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
Downloads · 30 days
0
Access
Public
Updated Nov 16, 2025
Repo size
117 MB
Likes
0
Public
Click a slice to open those files.
.safetensors113 MB · 96%
From the Hugging Face model README
This repository provides a LoRA fine-tuned version of
google/vit-large-patch16-224-in21k on ImageNet-1K.
本仓库提供了一个基于 google/vit-large-patch16-224-in21k 模型,使用 LoRA 技术在 ImageNet-1K 数据集上微调后的版本。
The model is optimized with low-rank adaptation (LoRA) while updating only ~2% of total parameters, achieving strong accuracy comparable to full fine-tuning. 模型使用 低秩适配 LoRA 技术进行高效训练,仅更新约 8% 参数量,并取得了接近甚至超越全量微调的效果。
| Metric 指标 | Score 分数 |
|---|---|
| Top-1 Accuracy | 82.66% |
| Top-5 Accuracy | 93.10% |
✔️ Achieved with early stopping ✔️ Training stability verified ✔️ Excellent compute-to-performance ratio
| Item 项目 | Value 配置 |
|---|---|
| Base Model | google/vit-large-patch16-224-in21k |
| Method | LoRA |
| LoRA Rank (r) | 64 |
| Trainable Params | 28,311,552 (trainable%: 8.5112) |
| Dataset | ImageNet-1K |
| Epochs | 20 (Early Stop at 16) |
| Optimizer | AdamW |
| Scheduler | CosineAnnealingWarmRestarts |
| Precision | AMP (FP16/BF16) |
| Input Resolution | 224×224 |
from transformers import ViTForImageClassification
from peft import PeftModel
import torch
device = "cuda" if torch.cuda.is_available() else "cpu"
base = ViTForImageClassification.from_pretrained(
"google/vit-large-patch16-224-in21k",
num_labels=1000
)
model = PeftModel.from_pretrained(base, "xn6o/lora-vit-large-patch16-224-in21k-r64-imagenet1k")
model.to(device).eval()
state_dict = torch.load("vit_classifier_best.pt", map_location=device)
model.base_model.classifier.load_state_dict(state_dict)
This step merges LoRA weights into the base model so no adapter is required at inference time. 此步骤将 LoRA 权重 永久合并,推理时无需 LoRA 适配器模块。
merged = model.merge_and_unload()
merged.save_pretrained("vit_lora_merged_best")
from PIL import Image
from torchvision import transforms
import torch.nn.functional as F
img = Image.open("your_image.jpg")
tfm = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406],
[0.229, 0.224, 0.225]),
])
pixel_values = tfm(img).unsqueeze(0).to(device)
with torch.no_grad():
logits = model(pixel_values=pixel_values).logits
probs = F.softmax(logits, dim=1)
top5 = probs.topk(5)
print("Top-5 Classes:", top5.indices.squeeze().tolist())
print("Top-5 Probabilities:", top5.values.squeeze().tolist())
|-- adapter/ # LoRA adapter weights
|-- vit_classifier.pt # Fine-tuned classifier head
|-- README.md # This model card
Follows original ViT model license: 遵循原始 ViT 模型的开源许可协议。
If this model is helpful, please star ⭐ the repo or open an issue! 如果这个模型对你有帮助,欢迎 点个 Star ⭐ 或提 Issue!