sam3.1-nvfp4-detector-no-language

SAM 3.1 NVFP4 — Detector (No Language), Static-6

NVFP4 quantized variant of facebook/sam3.1. Quantizes the detector backbone's vision trunk while keeping the language backbone in FP32. This is currently a storage-size quantization: the NomNomLabel loader dequantizes weights into the official SAM3 image model before execution; native packed-FP4 execution remains future work.

Quantization Details

Fidelity

Validated against source SAM 3.1 on mukbang / food video frames with Sapiens-2 human-part exclusion masking:

Some NVFP4 masks are visually sharper than the source FP32 output due to quantization-induced de-noising.

Known upstream loader warning

The current facebook/sam3.1 checkpoint also reports four missing backbone.vision_backbone.convs.3.* keys when loaded through the current SAM3 image-model builder. This warning is inherited from the upstream checkpoint / builder combination and is not introduced by the NVFP4 packing. The NomNomLabel loader suppresses this exact known upstream notice, while still surfacing any other missing-key output.

Files

Usage

# Load the quantized model using the nomnomlabel loader
from nomnomlabel.quant_loader import load_sam3_nvfp4

model = load_sam3_nvfp4("Reza2kn/sam3.1-nvfp4-detector-no-language")

# Segment food (with language prompt)
from nomnomlabel.sam3_food_classifier import SAM3FoodClassifier

classifier = SAM3FoodClassifier(model_id="Reza2kn/sam3.1-nvfp4-detector-no-language")
segments = classifier.segment_and_classify_food(image, conf_threshold=0.35)

for seg in segments:
    print(f"Food: {seg.food_type} (conf={seg.food_conf:.2f})")
    print(f"  Mask area: {seg.area} pixels")
    print(f"  BBox: {seg.bbox}")

Training / Benchmark Context

Built as part of the NomNomLabel Sapiens2 Benchmark for food instance segmentation in long-form mukbang / eating content. The teacher stack uses:

• SAM 3.1 NVFP4 (this model) for open-vocabulary food instance segmentation
• Sapiens-2 INT4-G128 for human-part segmentation and exclusion masking
• Sapiens-2 normals, pointmap/depth proxy, and matting as auxiliary human-scene teachers

Source

Base checkpoint: facebook/sam3.1.

---

Quantized by Reza2kn using torchao NVFP4.