Tetraopes_on_flowers

Tetraopes Beetle Flower Detection Model

Model Description

This model detects whether photographs of Tetraopes beetles (Cerambycidae: Tetraopini) contain milkweed flowers or fruits (Asclepias spp.). It is designed to facilitate large-scale analysis of iNaturalist observations to identify beetle-plant associations for ecological and co-phylogenetic studies.

Model Architecture: Vision Transformer Large (ViT-L/14) with CLIP pre-training

Base Model: vit_large_patch14_clip_336.openai_ft_in12k_in1k

• Pre-trained by OpenAI on 400M image-text pairs (CLIP)
• Fine-tuned on ImageNet-12k and ImageNet-1k
• Input resolution: 336×336 pixels

Task: Binary image classification (flower present: yes/no)

Training Framework: FastAI with PyTorch backend

Last Updated: 2025-11-07

Intended Use

Primary Use Case

This model is designed specifically for:

• Identifying Tetraopes beetle observations on iNaturalist that show beetles on milkweed flowers/fruits
• Supporting ecological studies of beetle-plant associations
• Enabling large-scale analysis of host plant usage patterns
• Filtering iNaturalist photos for downstream analysis (e.g., host plant identification)

Intended Users

• Ecologists studying plant-insect interactions
• Entomologists researching Tetraopes biology
• Researchers conducting co-phylogenetic analyses
• Biodiversity scientists working with iNaturalist data

Out-of-Scope Use

This model is NOT intended for:

• General flower detection in arbitrary contexts
• Detection of beetles other than Tetraopes
• High-stakes decision making
• Any application where misclassification could cause harm

Limitations

1. Taxonomic Scope: Trained specifically on Tetraopes beetles; may not generalize to other beetle genera
2. Plant Scope: Optimized for milkweed (Asclepias) flowers/fruits; performance on other plants not evaluated
3. Image Quality: Performance depends on image quality typical of iNaturalist photos
4. Confidence Calibration: Based on validation analysis, only predictions with >80% confidence are recommended for use
5. Small Training Set: Trained on only 121 manually labeled images; may not capture all variation
6. Class Imbalance: Training data is approximately balanced, but real-world distribution is unknown

Training Data

Dataset Summary

• Source: Manually labeled photos from iNaturalist observations
• Total Images: 119
  • With flowers (yes): 63
  • Without flowers (no): 56
• Train/Val Split: 80/20 stratified by class (seed=42)
  • Training: ~95 images
  • Validation: ~24 images
• Image Source: iNaturalist community observations

Training Images

All training and validation images are from iNaturalist observations and remain under their original licenses. Links to all training/validation photos are provided below.

Notes:

• For observations with multiple photos, the first photo was used for training/validation.
• Original quality images (full resolution) were downloaded from iNaturalist and then resized to 336×336 pixels during training.

Training/Validation Images WITH Flowers (n=63)

Training/Validation Images WITHOUT Flowers (n=56)

Training Procedure

Hyperparameters

• Model: Vision Transformer Large (ViT-L/14)

  • Architecture: vit_large_patch14_clip_336.openai_ft_in12k_in1k
  • Patch size: 14×14
  • Input resolution: 336×336 pixels

• Training Strategy: Transfer learning with fine-tuning

  • Frozen epochs: 3 (train only classification head)
  • Fine-tuning epochs: 3 (train all layers)
  • Total epochs: 6

• Optimization:

  • Optimizer: AdamW (fastai default)
  • Learning rate: Determined via lr_find() - valley of loss curve
  • Learning rate schedule: One-cycle policy
  • Batch size: 8 (per GPU)
  • Weight decay: 0.01 (fastai default)

• Hardware:

  • 2× NVIDIA RTX A5000 GPUs (24GB each)
  • DataParallel training for faster convergence

• Data Augmentation:

  • Geometric:
    • Random rotation: ±10°
    • Random zoom: 1.0-1.3×
    • Random scale: 0.8-1.0×
    • Random warp: ±0.2
    • Horizontal flip: 50%
    • Vertical flip: 50%
  • Photometric:
    • Random lighting: ±0.4
  • Advanced:
    • MixUp augmentation (α=0.4) - blends image pairs during training
  • Normalization: ImageNet statistics (mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

• Class Balancing:

  • Weighted sampling during training
  • Sample weights calculated using normalized inverse frequency
  • Formula: weight = total_samples / (num_classes × class_count)

Training Procedure Details

1. Data Preparation:

   • Original quality images downloaded from iNaturalist (full resolution, typically ~2048px)
   • Images resized to 336×336 using squish method (no cropping)
   • Applied augmentation transforms
   • Normalized with ImageNet statistics

2. Transfer Learning:

   • Phase 1 (3 epochs): Froze all layers except classification head
   • Phase 2 (3 epochs): Unfroze all layers for end-to-end fine-tuning
   • Used MixUp augmentation throughout training

3. Validation:

   • Evaluated on held-out 20% validation set
   • No augmentation applied during validation
   • Metrics: accuracy, error rate

Evaluation

Validation Results

• Validation Accuracy: 100% (25/25 images correct)
• Validation Error Rate: 0%

Confusion Matrix (Validation Set):

          Predicted No  Predicted Yes
Actual No           12              0
Actual Yes           0             13

Note: Perfect validation accuracy on a small dataset (25 images) suggests potential overfitting. Real-world performance is expected to be lower. Based on visual inspection of large-scale predictions, confidence thresholding (>80%) is recommended.

Large-Scale Prediction Results

The model was applied to 75,428 iNaturalist photos from 50,839 Tetraopes observations:

• Predictions:

  • Flower detected (yes): 31,419 (41.7%)
  • No flower (no): 44,009 (58.3%)
  • Mean confidence: 84.0%

• High-Confidence Results (yes prediction with >80% confidence):

  • Photos: 21,307 (28.2%)
  • Unique observations: 15,916
  • Recommendation: Only use predictions with >80% confidence based on manual validation

Performance by Taxon

Showing top 15 of 25 taxa

Notes:

• "High-Conf Photos" = predictions with >80% confidence showing flowers
• "High-Conf Obs" = observations with at least one high-confidence flower photo
• Some taxa have limited observations; interpret results with caution

How to Use

Requirements

pip install fastai timm torch torchvision pandas

Loading the Model

from fastai.vision.all import *
import torch

# Download model from Hugging Face
model_path = 'tetraopes_flower_vit_large_patch14_clip_336.openai_ft_in12k_in1k.pth'

# Create a dummy DataLoader with the same structure as training
# (FastAI requires this to load the model)
from fastai.data.all import DataBlock, ColSplitter, ColReader
import pandas as pd

# Create minimal dataframe for loading
dummy_df = pd.DataFrame({'path': ['dummy.jpg'], 'label': ['yes'], 'is_valid': [False]})

dbl = DataBlock(
    blocks=(ImageBlock, CategoryBlock),
    get_x=ColReader('path'),
    get_y=ColReader('label'),
    splitter=ColSplitter('is_valid'),
    item_tfms=Resize(336, method='squish'),
    batch_tfms=Normalize.from_stats(*imagenet_stats)
)

dls = dbl.dataloaders(dummy_df, bs=1)

# Create learner and load weights
learn = vision_learner(
    dls,
    'vit_large_patch14_clip_336.openai_ft_in12k_in1k',
    metrics=[error_rate, accuracy]
)
learn = learn.to_fp16()  # Use mixed precision
learn.load(model_path.replace('.pth', ''))  # Remove .pth extension

print("Model loaded successfully!")

Making Predictions

# Single image prediction
img_path = 'path/to/tetraopes_photo.jpg'
pred_class, pred_idx, probs = learn.predict(img_path)

print(f"Prediction: {pred_class}")
print(f"Confidence: {probs[pred_idx]:.2%}")
print(f"P(no flower): {probs[0]:.2%}")
print(f"P(with flower): {probs[1]:.2%}")

# Recommended: Filter by confidence threshold
confidence = probs[pred_idx]
if pred_class == 'yes' and confidence > 0.80:
    print("High-confidence flower detection!")
else:
    print("Low confidence or no flower detected")

Batch Predictions

from pathlib import Path

# Get all images
image_files = list(Path('images/').glob('*.jpg'))

# Create test DataLoader
test_dl = learn.dls.test_dl(image_files, bs=16)

# Get predictions
preds, _ = learn.get_preds(dl=test_dl)

# Process results
results = []
for idx, img_path in enumerate(image_files):
    pred_idx = preds[idx].argmax().item()
    pred_class = learn.dls.vocab[pred_idx]
    confidence = float(preds[idx][pred_idx])
    prob_no = float(preds[idx][0])
    prob_yes = float(preds[idx][1])

    results.append({
        'image': img_path.name,
        'prediction': pred_class,
        'confidence': confidence,
        'prob_no': prob_no,
        'prob_yes': prob_yes,
        'high_confidence_flower': pred_class == 'yes' and confidence > 0.80
    })

results_df = pd.DataFrame(results)
print(results_df)

Model Card Authors

Bruno A. S. de Medeiros

Citation

If you use this model in your research, please cite:

@misc{tetraopes_flower_detector_2025,
  author = {de Medeiros, Bruno A. S.},
  title = {Tetraopes Beetle Flower Detection Model},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/brunoasm/Tetraopes_on_flowers}},
  note = {Vision Transformer model for detecting Tetraopes beetles on milkweed flowers}
}

Related Repository: The full code for training this model and conducting co-phylogenetic analyses is available at: https://github.com/de-Medeiros-insect-lab/Tetraopes_cophylogeny_analyses

Additional Information

Framework Versions

• Python: 3.13.9
• PyTorch: 2.6.0
• FastAI: 2.8.5
• timm: 1.0.21

Model Size

• Parameters: ~304M (ViT-Large)
• Model file size: ~3.4 GB

Computational Requirements

• Training: 2× NVIDIA RTX A5000 (24GB) - ~30 minutes total
• Inference:
  • GPU recommended (any CUDA-capable GPU with 8GB+ VRAM)
  • CPU inference possible but slow (~1-2 seconds per image)
  • FP16 recommended for faster inference

License

This model is released under the Apache 2.0 license.

Important: The training images remain under their original iNaturalist licenses (typically CC-BY, CC-BY-NC, or CC0). Links to all training images with their original licenses are provided in the "Training Images" section above.

Contact

For questions or issues, please open an issue on the GitHub repository: https://github.com/de-Medeiros-insect-lab/Tetraopes_cophylogeny_analyses

Acknowledgments

• Training data from the iNaturalist community
• Pre-trained weights from OpenAI (CLIP) and timm library
• Training framework: FastAI