promac-aria-stage2-random-visfrozen

ProMAC Aria Stage-2 Random Init, Vision Frozen

Public, inference-only Qwen3-VL export of the ProMAC Aria Stage-2 structured-CoT v4 checkpoint trained with a randomly initialized future-state predictor and frozen Qwen vision tower.

Results

On the 1,051-example Aria structured-CoT v4 validation split (and-v4-val), deterministic evaluation reported:

The DeepSpeed tag is optimizer global_step1050, corresponding to training micro-step 2100 with gradient accumulation 2. Coordinates and errors are in pixels in the rotated 512 x 512 Aria frame. The goal is supplied in the prompt, so Aria FDE is partly degenerate.

Usage

import torch
from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

model_id = "wjdghks950/promac-aria-stage2-random-visfrozen"
processor = AutoProcessor.from_pretrained(model_id, fix_mistral_regex=True)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

Use the Aria structured-CoT v4 prompt schema and answer_only assistant format. Full offline evaluation used deterministic generation with up to 512 new tokens.

Scope

This repository contains the Qwen3-VL subtree extracted from the exact ProMAC ZeRO-3 checkpoint and merged into a standard Hugging Face model. Trajectory inference uses the standard Qwen generate() path. The randomly initialized predictor affected training through the auxiliary objective but is not needed for trajectory inference and is not included. The auxiliary decoder, SIGReg state, optimizer state, raw ZeRO shards, logs, and datasets are also excluded.

The export was merged with Qwen/Qwen3-VL-4B-Instruct revision ebb281ec70b05090aa6165b016eac8ec08e71b17. See provenance.json and SHA256SUMS.

License And Data Terms

The upstream Qwen model card declares Apache-2.0. No standalone license was established locally for the ProMAC fine-tuned weights, so this repository uses license: other. Access to these weights does not grant rights to the Aria training or evaluation data. Users are responsible for all applicable source-data terms.

Limitations

This is a research checkpoint selected on a limited training-time evaluation and later evaluated offline. It may not generalize to other prompt formats, camera geometry, rotations, environments, or trajectory schemas. Reported metrics are not safety guarantees.