Downloads · 30 days
0
chenliforai/mars2-saucy-aet-grpo-requeued
mars2-saucy-aet-grpo-requeued is a machine learning model from chenliforai. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
Weights for Team Saucy's best evaluated entry to the Video Temporal Grounding track of the MARS2 2026 Multimodal Reasoning Challenge. Score 0.5942413 (submission 578199).
Downloads · 30 days
0
Access
Public
Updated Aug 4, 2026
Repo size
11.1 GB
Likes
0
Public
Click a slice to open those files.
.safetensors11 GB · 100%
From the Hugging Face model README
Weights for Team Saucy's best evaluated entry to the Video Temporal Grounding track of the MARS2 2026 Multimodal Reasoning Challenge. Score 0.5942413 (submission 578199).
That submission was accepted before the deadline but only evaluated after the challenge went
inactive, so it is not shown on the public leaderboard; the displayed entry is submission
577915 at 0.5891353, whose adapter is published at
chenliforai/mars2-saucy-aet-grpo.
Code, reproduction instructions and the full pipeline:
https://github.com/chenlicodebank/mars2-saucy-eccv2026 (branch alternate-if-requeued-counted)
| Folder | What | Size |
|---|---|---|
base_stage1_merged/ | Qwen3-VL-4B-Instruct with our stage-1 Charades-STA GRPO adapter merged in. The adapter below attaches to this, not to the stock release. | 8.3 GB |
adapter_stage3_clockrepair/ | The deployed model. Stage-3 clock-repair LoRA adapter, step 65. Scored 0.5942413 (submission 578199). | 2.2 GB |
No MARS2 challenge data or predictions are included — the MARS2 2026 Data Use and Sharing License Agreement forbids redistributing the dataset or any derivative of it.
Qwen3-VL-4B-Instruct post-trained with Group Relative Policy Optimization on a pure temporal-IoU
reward, reading an Audio-Evidence Timeline: word-level ASR (faster-whisper large-v3,
rendered word-fine) and non-speech sound events (BEATs / AudioSet-Strong) as timestamped text
beside frames sampled at 2 fps under a whole-video budget of 14,680,064 pixels.
LoRA (r=64, alpha=16, dropout 0.05) is the only trainable component; the vision tower,
language model and merger are frozen. Three stages, all beta=0 (no KL), batch 1 × gradient
accumulation 8, 8 generations per prompt, bf16:
2e-6, seed 42 (6.5 h +
3.2 h on one GPU). Merged into the base to give base_stage1_merged/.2e-6, total_tokens 17408 (9.1 h on one GPU).5e-6, DDP
across 3 GPUs with a fourth reserved as an evaluation meter, checkpointing every five steps.
Step 65 is the argmax on a held-out human-labelled meter (2.1 h wall-clock ≈ 6.2 GPU-h). This
is adapter_stage3_clockrepair/.Total training cost 18.9 + 6.2 GPU-hours. One full pass over the 3,108-item test set took 55 min 4-way across 4 GPUs at 4.3 s/video (≈3.7 GPU-h).
The clock-repair stage is the interesting one: the advertisement transcripts had been reaching
the model on a coarser clock than the gold spans are defined against, so the model was being
rewarded for quoting boundaries it could not actually see. Fixing the clock and re-training for
65 steps was worth +0.0051, more than any architectural or decoding change we tried after the
recipe settled.
from transformers import Qwen3VLForConditionalGeneration
from peft import PeftModel
REPO = "chenliforai/mars2-saucy-aet-grpo-requeued"
base = Qwen3VLForConditionalGeneration.from_pretrained(
REPO, subfolder="base_stage1_merged", dtype="bfloat16", device_map="cuda")
model = PeftModel.from_pretrained(base, REPO, subfolder="adapter_stage3_clockrepair")
model.eval() # do NOT call merge_and_unload()
merge_and_unload() computes W + BA*scale and casts the result back to bf16. For this rank-64
adapter, 97.3% of the trained delta does not survive that cast — the elementwise update is
smaller than the bf16 ulp of the corresponding base weight. Keeping the adapter as a live
low-rank branch is worth +0.001 on the leaderboard over the merged copy of the same checkpoint.
Every number we reported was produced with --no-merge-adapter.
Note that target_modules includes embed_tokens and lm_head, which is why the adapter is
larger than a typical rank-64 LoRA.
The model expects the timeline in the prompt, not raw audio. Build the ASR and SED stores with
scripts/asr_mars2.py and scripts/sed_mars2.py from the code repository, then run
scripts/run_zeroshot_vtg.py --asr-format word-fine. The exact prompt construction lives in that
script; a mismatched prompt format costs several points, because the model was trained inside one.
Please cite the MARS2 2026 challenge report. Underlying components: Qwen3-VL, GRPO (DeepSeekMath), LoRA, Whisper, PretrainedSED, Charades-STA, AdsQA, and the TimeLens GRPO trainer, which this work vendors.
We acknowledge the use of the MARS2 Dataset provided by the MARS2 2026 Challenge (https://eval.ai/web/challenges/challenge-page/4688/overview).