project_azoth

AZOTH — Open-Vocabulary Object Detection (ANIMA Module)

Part of the ANIMA Perception Suite by Robot Flow Labs.

Model

AZOTH is a real-time open-vocabulary object detection module built on LLMDet (CVPR 2025). It detects arbitrary objects from natural language queries without predefined class vocabularies.

• Architecture: GroundingDINO + Swin-T backbone + BERT text encoder
• Variant: LLMDet-Tiny (172.9M parameters)
• Input: RGB image + text query (natural language)
• Output: Bounding boxes + confidence scores

Paper

LLMDet: Learning to Understand Visual Grounding with LLMs arXiv: 2602.05730 | CVPR 2025

Evaluation — LVIS minival

This baseline exceeds the paper's reported AP on LVIS minival.

Exported Formats

Usage

from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor
from PIL import Image

model = AutoModelForZeroShotObjectDetection.from_pretrained(
    "ilessio-aiflowlab/project_azoth",
    subfolder="pytorch"
)
processor = AutoProcessor.from_pretrained(
    "ilessio-aiflowlab/project_azoth",
    subfolder="pytorch"
)

image = Image.open("test.jpg")
inputs = processor(images=image, text="a person. a car. a dog.", return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)

results = processor.post_process_grounded_object_detection(
    outputs, inputs.input_ids, threshold=0.3, target_sizes=[(image.height, image.width)]
)

Training Data

Trained on GroundingCap-1M (CVPR 2025):

• COCO 2017 (205K samples)
• GQA (358K samples)
• LLaVA-ReCap-558K (307K samples)
• Flickr30k (79K samples)
• V3Det (166K samples)

Training Config

• Optimizer: AdamW, weight_decay=0.05
• LR: 1e-4, cosine decay
• Warmup: 1000 iterations
• Epochs: 12 (1x schedule)
• Batch size: 16 (8 GPUs x 2)
• Mixed precision: FP16
• Hardware: 8x NVIDIA L4 (23GB each)

Directory Structure

pytorch/          SafeTensors + PyTorch weights + tokenizer
checkpoints/      Best training checkpoint (resume training)
configs/          Training TOML configs
reports/          LVIS evaluation reports (JSON)
paper.pdf         LLMDet CVPR 2025 paper

License

Apache 2.0 — Robot Flow Labs / AIFLOW LABS LIMITED