Downloads · 30 days
0
xn6o/lora-vit-large-patch16-224-in21k-r96-imagenet1k
lora-vit-large-patch16-224-in21k-r96-imagenet1k is a machine learning model from xn6o. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
This repository contains a LoRA finetuned version of google/vit-large-patch16-224-in21k for ImageNet-1K classification, using rank = 96 LoRA adapters with additional data augmentation and strong regularization. The cl…
Downloads · 30 days
0
Access
Public
Updated Nov 17, 2025
Repo size
174 MB
Likes
0
Public
Click a slice to open those files.
.safetensors170 MB · 98%
From the Hugging Face model README
This repository contains a LoRA finetuned version of google/vit-large-patch16-224-in21k for ImageNet-1K classification, using rank = 96 LoRA adapters with additional data augmentation and strong regularization.
The classifier head was retrained from scratch and is provided separately.
| Property | Value |
|---|---|
| Base Model | google/vit-large-patch16-224-in21k |
| Task | Image Classification (ImageNet-1K) |
| Adapter Type | LoRA |
| LoRA Rank | 96 |
| Trainable Params | ~42M |
| Total Params | ~305M |
| Trainable Ratio | ~12.24% |
| Best Val Top-1 | 82.82% |
| Best Val Top-5 | 93.93% |
| Filename | Description |
|---|---|
adapter_model.safetensors | LoRA weights |
adapter_config.json | Configuration for LoRA |
vit_classifier_r96.pt | Full classifier layer (1000 classes) |
This LoRA model does not include classifier weights in the adapter.
To reproduce correct evaluation accuracy, you MUST load both:
1️⃣ Base model
2️⃣ LoRA adapter
3️⃣ Custom classifier parameters
Failure to load the classifier will significantly reduce accuracy.
import torch
from transformers import ViTForImageClassification
from peft import PeftModel
device = "cuda" if torch.cuda.is_available() else "cpu"
# 1. Load base model
model = ViTForImageClassification.from_pretrained(
"google/vit-large-patch16-224-in21k",
num_labels=1000
)
# 2. Load LoRA adapter
model = PeftModel.from_pretrained(model, "xn6o/lora-vit-large-patch16-224-in21k-r96-imagenet1k")
# 3. Load trained classifier weights
state_dict = torch.load("vit_classifier_r96.pt", map_location=device)
model.classifier.load_state_dict(state_dict)
model.to(device)
model.eval()
print("Model fully loaded with LoRA + classifier.")
with torch.no_grad():
outputs = model(pixel_values=images)
preds = torch.argmax(outputs.logits, dim=-1)
Key techniques:
This model follows the same license as the original ViT base model from Google Research and Hugging Face Transformers. Usage must comply with ImageNet dataset license.
---
- PEFT 0.17.1