Downloads · 30 days
3
4% of all-time downloads
ohjoonhee/vlatents-qwen25vl7b-stage2-repro-v2
vlatents-qwen25vl7b-stage2-repro-v2 is a image-text-to-text model from ohjoonhee. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
One-line identity: Stage 2 reproduction of Monet-style visual-CoT training on Qwen/Qwen2.5-VL-7B-Instruct. Initialised from a Stage 1 SFT base; trained on the ohjoonhee/visual-cot-50k-poc dataset.
Downloads · 30 days
3
4% of all-time downloads
All-time downloads
78
Public
Parameters
8.3B
16.6 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.6 GB · 100%
From the Hugging Face model README
One-line identity: Stage 2 reproduction of Monet-style visual-CoT training on Qwen/Qwen2.5-VL-7B-Instruct. Initialised from a Stage 1 SFT base; trained on the ohjoonhee/visual-cot-50k-poc dataset.
Qwen/Qwen2.5-VL-7B-Instruct/data/joonhee/visual-latents/cluster_phase3/stage1_sft/checkpointohjoonhee/visual-cot-50k-poc (Monet-SFT-125K Visual_CoT subset, eval-200 excluded)latent_size: 8alignment_weight: 2.0emphasize_latent_weight: 2.0ce_emphasize_factor: 4.0alignment_layer: all_layersuse_attn_mask_4d: Truelr: 1e-05weight_decay: 0.01warmup_steps: 100max_steps: 2000grad_accum_steps: 16max_pixels: 784000emphasize_latent_weight
uses a verbatim port of upstream compute_latents_only_loss: the alignment
loss is computed in the CE forward (where ce_patch_vec is spliced into
inputs_embeds) and backpropped ONLY through the latent embeddings, i.e.
total = emphasize_latent_weight * compute_latents_only_loss(ce_patch_vec, alignment_weight*align) + ce (mirrors upstream src/trainer.py:152-224).
The earlier plain-scalar-add approximation (see the *-repro-v1 repo) is
NOT used here.attention_mask_4d is hand-rolled in mask_utils.build_monet_4d_attn
with latent_cross_isolate=True. Verified equivalent on tested cases
(see phase1_5b_attn/MASK_VALIDATION.md) but not byte-identical to upstream.step-1500)Last logged training row: step=1690, ce_loss=0.7810, align_loss=0.4884
Job C: deviation #1 (latent-only backprop) is paper-faithful - verbatim upstream compute_latents_only_loss (see Fidelity section). Smoke-verified latent_only_connected=True (n_nonzero_grad=1). Training walltime-truncated near step 1690/2000; this is the step-1500 checkpoint. ce0.8 align0.46 stable throughout.
Other revisions: see the revisions dropdown on this page.
from transformers import AutoModelForVision2Seq, AutoProcessor
m = AutoModelForVision2Seq.from_pretrained(
"ohjoonhee/vlatents-qwen25vl7b-stage2-repro-v2", revision="step-1500", torch_dtype="bfloat16")
p = AutoProcessor.from_pretrained("ohjoonhee/vlatents-qwen25vl7b-stage2-repro-v2", revision="step-1500")
Research checkpoint, eval-only. Mid-training step (1500/2000). Not for production.
Card generated 2026-05-19 from training_log.jsonl + the run's training config.