DipViT

Dip-ViT — Forensically Relevant Insect Classification

This repository contains the models, datasets, and application files for the Research Project: A Comparative Evaluation of Vision Transformer and Convolutional Neural Network Performance for Forensically Relevant Dipteran Classification

Author: Luka Kowalchuk

University: University of Amsterdam

Year: 2025–2026

Student number: 13326015

GitHub featuring all scripts: https://github.com/LukaKowalchuk/Dip-ViT

Project Overview

This work aimed to support forensic entomology investigations by accelerating insect identification, a key component in postmortem interval estimation and wildlife crime investigations. The main backbone architecture is BioCLIP, a Vision Transformer pretrained on images of the biological domain, finetuned using Low Rank Adaptation (LoRA) on a database developed by the Wildlife Forensic Academy. The research demonstrated that Vision Transformers outperformed a previously developed CNN, while also identifying important limitations in the existing insect database. Explainable AI techniques were explored to improve model interpretability, and a Streamlit-based demonstration application was developed for real-world implementation.

Repository Contents

streamlit_files/

All files required to run the Streamlit demonstration app:

BioCLIP-1_lora_best_flies.pt / BioCLIP-2_lora_best_flies.pt — best LoRA finetuned models for adult fly classification

BioCLIP-1_lora_best_maggots.pt / BioCLIP-2_lora_best_maggots.pt — best LoRA finetuned models for maggot classification

class_names.json — class label mappings for each model/mode combination

train_embeddings.npy / train_labels.npy — precomputed KNN embeddings and labels for open-set recognition

WFA-logo.webp, uvalogo_regular_p_en.jpg — application assets

datasets/

final_dataset.zip — current dataset used for training and evaluation, containing ~5000 images across forensically relevant Dipteran species (Chrysomya albiceps, C. marginalis, C. chloropyga, Lucilia sericata/cuprina, Muscidae, M. Synthesiomya nudiseta), for both adult flies and maggots

final_dataset_old_version.zip — original dataset from the preceding project, retained for reproducibility

Contact

Email: [email protected]

LinkedIn: https://www.linkedin.com/in/luka-kowalchuk-186092222/