OpenPlants-Identification-ViT-Base-Patch16-224

<div align="center"> <img src="openplants.png" alt="OpenPlants logo" width="220" /> <h1>OpenPlants Plant Identification Model</h1> <p>Species-level plant classifier for <a href="https://github.com/domai-tb/OpenPlants">OpenPlants</a>.</p> </div>

This model is used in OpenPlants (domai-tb/OpenPlants), an open-source and privacy-friendly companion app for plant care. Inside the app, it powers local image classification so users can identify plants directly on-device from a photo.

Built on google/vit-base-patch16-224, the model was fine-tuned for species-level plant identification on the GBIF / iNaturalist plant image dataset.

Highlights

OpenPlants

OpenPlants is the companion app this model was built for.

• Runs plant identification locally
• Keeps inference on-device for a privacy-friendly experience
• Helps users identify plants in a fast, offline-friendly workflow

Example Usage

from transformers import AutoImageProcessor, AutoModelForImageClassification
from PIL import Image
import requests
import torch

model_id = "domai-tb/OpenPlants-Identification-ViT-Base-Patch16-224"

processor = AutoImageProcessor.from_pretrained(model_id)
model = AutoModelForImageClassification.from_pretrained(model_id)

url = "https://example.com/plant.jpg"
image = Image.open(requests.get(url, stream=True).raw)

inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
    logits = model(**inputs).logits

probs = logits.softmax(dim=-1)[0]
topk = torch.topk(probs, k=5)

for prob, idx in zip(topk.values, topk.indices):
    label = model.config.id2label[idx.item()]
    print(f"{label}: {prob.item():.4f}")

Intended Applications

• Plant identification in mobile apps
• Ecological surveys
• Nursery and horticulture tools
• Restoration and revegetation workflows
• Field research and biodiversity monitoring
• Citizen science and educational platforms
• Image-based species tagging pipelines

Data and Training Details

Dataset Construction

• Sourced from GBIF / iNaturalist occurrences with valid species and image metadata
• Cleaned and deduplicated before training
• Species filtered to those with at least 20 images
• Maximum cap of 1,000 images per species to reduce class imbalance
• Final training dataset: 2,000,000 images across roughly 14,000 species

Training

• ViT-Base fine-tuned for 1 epoch over 2M samples
• AdamW optimizer with standard ViT augmentations
• Mixed-precision training on GPU

Limitations

• Some species are visually indistinguishable without context such as location or reproductive structures
• Performance varies for rare, morphologically similar, or poorly photographed species
• The model is purely image-based and does not use location metadata

Labels

Species names follow canonical GBIF taxonomy (species_name). Each class maps directly to one species.

You can inspect all labels with:

from transformers import AutoConfig

cfg = AutoConfig.from_pretrained("domai-tb/OpenPlants-Identification-ViT-Base-Patch16-224")
labels = cfg.id2label

Fine-Tuning and Adapters

You can further specialize the model with LoRA adapters for:

• Regional subsets
• Functional groups
• Threatened species
• Agricultural crops
• Disease classification

The base model is broad enough to support domain-specific adapter tuning with relatively little compute.

License

Model weights follow the same license as the underlying ViT-Base model. Users are responsible for complying with GBIF and iNaturalist usage terms for any downstream dataset creation.