plant-disease-clip

Plant Disease CLIP Model (Fine-tuned ViT-B-32)

This is a fine-tuned CLIP ViT-B-32 model for plant disease recognition.

Model Details

• Base Model: OpenAI CLIP ViT-B-32
• Fine-tuning Method: Supervised Contrastive Learning with Regularization
• Dataset: 38 plant disease classes
• Training Samples: 70,296
• Validation Samples: 17,572
• Embedding Dimension: 512

Training Details

• Epochs: 4
• Best Validation Loss: 4.7171
• Fine-tuning Strategy: Light fine-tuning with regularization to preserve generalization
• Regularization: MSE loss to keep embeddings close to original CLIP

Classes

The model was trained on the following plant disease classes:

• Apple___Apple_scab
• Apple___Black_rot
• Apple___Cedar_apple_rust
• Apple___healthy
• Blueberry___healthy
• Cherry_(including_sour)___Powdery_mildew
• Cherry_(including_sour)___healthy
• Corn_(maize)___Cercospora_leaf_spot Gray_leaf_spot
• Corn_(maize)__Common_rust
• Corn_(maize)___Northern_Leaf_Blight
• Corn_(maize)___healthy
• Grape___Black_rot
• Grape___Esca_(Black_Measles)
• Grape___Leaf_blight_(Isariopsis_Leaf_Spot)
• Grape___healthy
• Orange___Haunglongbing_(Citrus_greening)
• Peach___Bacterial_spot
• Peach___healthy
• Pepper,_bell___Bacterial_spot
• Pepper,_bell___healthy
• Potato___Early_blight
• Potato___Late_blight
• Potato___healthy
• Raspberry___healthy
• Soybean___healthy
• Squash___Powdery_mildew
• Strawberry___Leaf_scorch
• Strawberry___healthy
• Tomato___Bacterial_spot
• Tomato___Early_blight
• Tomato___Late_blight
• Tomato___Leaf_Mold
• Tomato___Septoria_leaf_spot
• Tomato___Spider_mites Two-spotted_spider_mite
• Tomato___Target_Spot
• Tomato___Tomato_Yellow_Leaf_Curl_Virus
• Tomato___Tomato_mosaic_virus
• Tomato___healthy

Usage

import torch
import open_clip
from PIL import Image

# Load the model
model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32', pretrained='openai')
checkpoint = torch.load('model.pt', map_location='cpu')
model.visual.load_state_dict(checkpoint['vision_encoder'])
model.eval()

# Process an image
image = preprocess(Image.open('plant_image.jpg')).unsqueeze(0)
with torch.no_grad():
    image_features = model.visual(image)
    image_features = image_features / image_features.norm(dim=-1, keepdim=True)

# Use embeddings for similarity search in your vector database

Intended Use

This model is designed for:

• Embedding plant disease images for similarity search
• Building a vector database for plant disease diagnosis
• Assisting farmers in identifying crop diseases

Limitations

• Trained on 38 specific plant disease classes
• Performance on completely unseen plant types may vary
• Should be used as a decision support tool, not a replacement for expert diagnosis

Training Data

The model was fine-tuned on the Plant Disease Dataset containing images of:

• Apple, Blueberry, Cherry, Corn, Grape, Orange, Peach, Pepper, Potato, Raspberry, Soybean, Squash, Strawberry, and Tomato plants
• Various disease conditions and healthy samples

Citation

If you use this model, please cite the original CLIP paper:

@inproceedings{radford2021learning,
  title={Learning Transferable Visual Models From Natural Language Supervision},
  author={Radford, Alec and Kim, Jong Wook and Hallacy, Chris and Ramesh, Aditya and Goh, Gabriel and Agarwal, Sandhini and Sastry, Girish and Askell, Amanda and Mishkin, Pamela and Clark, Jack and others},
  booktitle={International Conference on Machine Learning},
  pages={8748--8763},
  year={2021},
  organization={PMLR}
}