gemma3n-e2b-cddm-finetune

🌱 AgriVision Gemma

Vision Language Fine Tuning for Crop Disease Diagnosis This repository hosts the adapters and inference code for a vision-language model fine-tuned on agricultural data using QLoRA (via Unsloth). The project is actively in progress.

---

🔍 Model Description

Fine-tuning of the Gemma-3n-e2b-it model to build a multimodal assistant capable of understanding leaf images and conversational input for accurate crop disease diagnosis.

---

🛠 Intended Use & Limitations

Intended Use:

• Diagnose crop diseases from leaf images in chatbot-style interaction.
• Assist researchers and developers in agricultural AI applications.

Limitations:

• Model is fine-tuned on CDDM dataset; generalization to new crop species or disease types is unverified.
• Model is experimental and still under training—expected performance may change.

---

📦 Installation

pip install unsloth datasets transformers huggingface_hub timm

---

💻 Usage Example

from huggingface_hub import login
from unsloth import FastVisionModel, FastModel
from transformers import AutoProcessor

login("YOUR_HF_TOKEN")

processor = AutoProcessor.from_pretrained("USER/AgriVision-Gemma")
model = FastVisionModel.from_pretrained("USER/AgriVision-Gemma")

# Prepare conversation
messages = [
  {"role":"system","content":[{"type":"text","text":"You are a crop disease diagnosis assistant."}]},
  {"role":"user","content":[{"type":"image","image": my_leaf_image}, {"type":"text","text":"What disease on this leaf?"}]}
]
inputs = processor(text=processor.apply_chat_template(messages), images=[my_leaf_image], return_tensors="pt").to(model.device)
output = model.generate(**inputs)
print(output)

---

⚙️ Training Details (Ongoing)

• Uses FastModel with 4-bit quantization and LoRA adapters configured on all-linear layers.
• Trained using Trainer with 8-bit AdamW, gradient checkpointing, and batched conversational inputs.
• Checkpointing enabled and interim adapter updates pushed to Hugging Face Hub.

---

🧰 Tech Stack

Frameworks: PyTorch, Transformers, Unsloth Libraries: datasets, PIL, timm, numpy, huggingface_hub

---

📘 How to Contribute

We welcome community feedback and improvements:

1. Fork the repository and submit adapter updates or preprocessing improvements.
2. Open an issue to suggest features, datasets, or evaluation benchmarks.

---

📄 License

This model is released under the MIT License.

---

📟 Framework Versions

• PEFT 0.16.0
• Transformers 4.53.2
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.2