promac-futloc-stage1b-cot-trajectory

ProMAC FutLoc Stage-1b CoT + Trajectory

Public, inference-only Qwen3-VL export of the ProMAC FutLoc Stage-1b structured-CoT v4 checkpoint selected by validation ADE.

Results

On all 3,016 examples of the FutLoc structured-CoT v4 goal-overlay validation split (futloc-v4-val), deterministic evaluation reported:

FutLoc coordinates are pixels in the 1280 x 720 frame over 40 waypoints.

Usage

import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "wjdghks950/promac-futloc-stage1b-cot-trajectory"
processor = AutoProcessor.from_pretrained(model_id, fix_mistral_regex=True)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

Use the FutLoc structured-CoT v4 goal-only-overlay prompt schema and answer_only assistant format. Set max_new_tokens=1024; 512 tokens truncated trajectories in the full offline evaluation.

Scope

This repository contains the merged Qwen3-VL inference model and processor/tokenizer. Trajectory inference uses the standard Qwen generate() path. It intentionally excludes the ProMAC future-state predictor, auxiliary reasoning decoder and visual adapter, SIGReg module, optimizer/scheduler state, raw .pt checkpoint, logs, and datasets. It is not a resumable ProMAC training checkpoint.

The Stage-1b partial checkpoint was merged onto Qwen/Qwen3-VL-4B-Instruct revision ebb281ec70b05090aa6165b016eac8ec08e71b17. See provenance.json and SHA256SUMS.

License And Data Terms

The upstream Qwen model card declares Apache-2.0. No standalone license was established locally for the ProMAC fine-tuned weights, so this repository uses license: other. Ego4D data is distributed through a license-gated workflow; access to these weights does not grant rights to source data or derived imagery.

Limitations

This model was evaluated on the documented FutLoc structured-CoT v4 validation split and may not generalize to other cameras, prompts, environments, or trajectory schemas. Reported metrics are offline evaluation results, not safety guarantees.