Downloads · 30 days
6
18% of all-time downloads
bi199797/VR-egodex-qwen36-27b-lora-boundary
VR-egodex-qwen36-27b-lora-boundary is a machine learning model from bi199797. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as other.
LoRA adapter for Qwen/Qwen3.6-27B (qwen35 architecture, hybrid GatedDeltaNet + full-attention), fine-tuned to predict only the temporal segment boundaries of an egocentric video -- MM:SS.d - MM:SS.d per action segment…
Downloads · 30 days
6
18% of all-time downloads
All-time downloads
33
Public
Repo size
821 MB
Likes
0
Public
Click a slice to open those files.
.safetensors801 MB · 98%
From the Hugging Face model README
LoRA adapter for Qwen/Qwen3.6-27B (qwen3_5 architecture,
hybrid GatedDeltaNet + full-attention), fine-tuned to predict only the temporal segment
boundaries of an egocentric video -- MM:SS.d - MM:SS.d per action segment, one line per
segment, no hand-action text at all.
Same base model, data and video settings as
bi199797/VR-egodex-narration-qwen36-27b-lora-vision12-lr1e5,
but trained on a text-free target built by stripping the hand-action captions back out of
that model's training data (stage2_1_timestamp_only/scripts/build_boundary_manifest.py):
this model localizes when actions change without describing what they are. It is a
video-only baseline for temporal localization -- compare it against the sibling
caption-boundary model,
which is given the action text and only has to re-time it; conditioning on text takes the
in-loop timestamp score from 0.6048 to 0.6868 on the same eval sample (see below).
Training code: VR-finetune-VLM, stage2_1_timestamp_only/.
| Base weights | bf16 (full precision, not quantized) |
| Adapter | LoRA, rsLoRA scaling |
lora_r | 32 |
lora_alpha | 64 |
lora_dropout | 0.05 |
| Target modules | language-model attention/MLP only (q/k/v/o_proj, gate/up/down_proj) -- no vision-tower adaptation |
| Trainable params | ~200.3M / 27.6B (0.73%) |
| Data | 19,672 EgoDex episodes (stage2_1_timestamp_only/data/manifests/train.jsonl), boundaries derived from the same human-QA-corrected SRT captions as the narration model, text discarded |
| Prompt | stage1/prompts/boundary/v1.txt -- "identify only the timestamps where the person's action changes" |
| Epochs | 2 (1,230 steps) |
| Batch size | 1 x 16 grad-accum = 16 effective |
| Learning rate | 1e-4, cosine schedule, warmup 3% |
| Weight decay | 0.01 |
| Sampling | fps=2.0, max_frames=128, max_pixels=272384/frame |
| Max sequence length | 40,960 |
| Precision | bf16 |
| Cluster | 2x NVIDIA H100 80GB (device_map=auto) |
timestamp axis: cue-boundary IoU only, no text)In-loop eval logged during training (stage2/train.py's periodic callback, 24-episode sample
of stage2_1_timestamp_only/data/manifests/val.jsonl) -- not yet the full 300/549-episode
generation sweep the narration cards above report.
| step | timestamp_score | eval_loss |
|---|---|---|
| 100 | 0.5638 | 0.1612 |
| 600 | 0.6024 | 0.1594 |
| 1000 | 0.5971 | 0.1522 |
| 1230 (final) | 0.6048 | 0.1523 |
A parallel r64 run (same data/prompt, lora_r=64) finished at 0.5994 -- no better, so r32 is
the checkpoint uploaded here.
from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor
base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.6-27B", dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(base, "bi199797/VR-egodex-qwen36-27b-lora-boundary")
processor = AutoProcessor.from_pretrained("bi199797/VR-egodex-qwen36-27b-lora-boundary")
Prompt with the system prompt at stage1/prompts/boundary/v1.txt and a video; the model
returns one MM:SS.d - MM:SS.d line per action segment, first line starting at 00:00.0,
last line ending at the video's final frame.