Downloads · 30 days
58
36% of all-time downloads
mrdbourke/food-not-food-classifier-nextvit-v1
food-not-food-classifier-nextvit-v1 is a image classification model from mrdbourke. Use it when you need a label for an image. It is set up for timm. The card lists the license as apache-2.0.
CoreML deployable — CNN+ViT hybrid, best for on-device inference (iPhone/iPad)
Downloads · 30 days
58
36% of all-time downloads
All-time downloads
159
Public
Parameters
30.8M
123 MB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors123 MB · 100%
From the Hugging Face model README
CoreML deployable — CNN+ViT hybrid, best for on-device inference (iPhone/iPad)
A binary image classifier for detecting food and drink in images, trained via knowledge distillation from SigLIP2-so400m zero-shot labels on 3.1M images from DataComp-1B-food-and-drink-3M.
Part of a 3-model portfolio for the Nutrify food tracking pipeline.
| Field | Value |
|---|---|
| Architecture | nextvit_small.bd_ssld_6m_in1k_384 |
| Parameters | 30.7M |
| Input size | 384x384px |
| Labels | food_or_drink (0), not_food_or_drink (1) |
| FoodVision accuracy | 0.9220 |
| FoodVision F1 | 0.9472 |
| Training val accuracy | 0.9300 (epoch 5/5) |
| Throughput | 1156.0 img/s (batch 64, RTX 4090) |
import timm
import torch
from PIL import Image
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
# Load model
model = timm.create_model("nextvit_small.bd_ssld_6m_in1k_384", pretrained=False, num_classes=2)
weights = hf_hub_download("mrdbourke/food-not-food-classifier-nextvit-v1", "model.safetensors")
model.load_state_dict(load_file(weights))
model.eval()
# Prepare transform (use timm's built-in config)
from timm.data import resolve_data_config, create_transform
data_config = resolve_data_config(model.pretrained_cfg)
data_config["input_size"] = (3, 384, 384)
transform = create_transform(**data_config, is_training=False)
# Classify
image = Image.open("photo.jpg").convert("RGB")
input_tensor = transform(image).unsqueeze(0)
with torch.inference_mode():
logits = model(input_tensor)
probs = torch.softmax(logits, dim=1)
pred = logits.argmax(dim=1).item()
labels = ["food_or_drink", "not_food_or_drink"]
print(f"{labels[pred]}: {probs[0][pred]:.1%}")
These models were trained together as part of the Nutrify food/not_food classifier portfolio. Pick the right one for your use case:
| Model | Role | FV Accuracy | FV F1 | Params | Throughput | Repo |
|---|---|---|---|---|---|---|
| siglip2_base_256 | Highest accuracy | 91.3% | 94.1% | 92.9M | 2099.9 img/s | link |
| csatv2_11m | Fastest throughput | 92.2% | 94.7% | 10.7M | 4639.1 img/s | link |
| nextvit_small_384 | CoreML deployable | 92.2% | 94.7% | 30.7M | 1156.0 img/s | link |
Evaluated on 153,911 human-labeled images from the Nutrify FoodVision dataset (118K food + 35K not_food). This is an out-of-distribution test — the model was trained on DataComp-1B web images, not FoodVision images.
| Metric | Value |
|---|---|
| Accuracy | 0.9220 |
| F1 | 0.9472 |
| Precision | 0.9903 |
| Recall | 0.9078 |
| Total samples | 153,911 |
| Correct | 141,899 |
| Wrong | 12,012 |
This model is part of the Nutrify VLM pipeline — a cascading filter system for building a food/drink image dataset from billion-scale web crawls:
| Batch Size | img/s | VRAM (MB) |
|---|---|---|
| 64 | 1156.0 | 10912.5 |
| 128 | 1110.6 | 2451.3 |
| 256 | 1096.7 | 4376.5 |
| 512 | 1092.4 | 8226.9 |
Peak: 1156.0 img/s at batch 64
@misc{food-not-food-nextvit-small-384-v1,
author = {Daniel Bourke},
title = {Food/Not Food Classifier — nextvit_small_384 v1},
year = {2026},
publisher = {HuggingFace},
url = {https://huggingface.co/mrdbourke/food-not-food-classifier-nextvit-v1}
}