locateanything_3b

See our collection for all versions of LocateAnything.

Run LocateAnything with Keras 3: JAX, PyTorch, or TensorFlow

zeromodels/locateanything_3b

Paper: LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding (arXiv:2605.27365) · HF Papers

LocateAnything is a vision-language grounding model for detection, referring, pointing, layout, GUI/text grounding, and OCR. Build the instruction with locate_prompt(task, text), then parse boxes / points / grounding from the generated token ids.

For more details on the model, please go to the upstream model card.

Pure-Keras 3 conversion of nvidia/LocateAnything-3B for zeromodels. One implementation runs unmodified on TensorFlow / Torch / JAX.

This is a grounding VLM checkpoint (LocateAnythingConditionalGenerate). Prefer load_dtype="bfloat16".

✨ Quick start (open-vocabulary detection)

import os
os.environ["KERAS_BACKEND"] = "torch"  # or "jax" / "tensorflow"

import keras
import numpy as np
from PIL import Image
from zeromodels.models.locateanything import (
    LocateAnythingConditionalGenerate,
    LocateAnythingProcessor,
    locate_prompt,
)

model = LocateAnythingConditionalGenerate.from_weights(
    "zeromodels/locateanything_3b", load_dtype="bfloat16"
)
processor = LocateAnythingProcessor.from_weights("zeromodels/locateanything_3b")

image = Image.open("your_image.jpg").convert("RGB")
# Tasks: detection | referring | phrase_grounding | pointing |
# layout | text_grounding | OCR
prompt = locate_prompt("detection", "zebra")
inputs = processor(
    conversation=[
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": prompt},
            ],
        }
    ]
)
out = model.generate(
    **inputs, max_new_tokens=192, tokenizer=processor.tokenizer
)
ids = np.asarray(keras.ops.convert_to_numpy(out))[0].tolist()
boxes = processor.tokenizer.parse_boxes(ids)  # [0, 1000] grid
print(len(boxes), boxes[:2])

Load any LocateAnything variant the same way with from_weights("zeromodels/<variant>"):

Tips

• Set KERAS_BACKEND before importing Keras / zeromodels.
• Use parse_boxes for detection, parse_points for pointing, parse_grounding for referring / layout / text / OCR.
• Boxes and points are on a [0, 1000] grid; scale to pixels yourself.
• See LocateAnything docs and Loading Weights.
• Community / upstream safetensors still work via the hf: prefix, e.g. LocateAnythingConditionalGenerate.from_weights("hf:nvidia/LocateAnything-3B").

Special Thanks

A huge thank you to the NVIDIA LocateAnything authors for creating and releasing these models.

License: NVIDIA License (non-commercial / research). See the upstream card for component licenses (Qwen2.5, MoonViT, etc.).