orena-frame-weights

orena-frame-weights

LoRA adapters for the FRAME track of the ORena SAVE FOCUS Challenge (MICCAI 2026).

Setting (all checkpoints)

• Base models: Qwen/Qwen3.5-9B (9B) and Qwen/Qwen3.8-27B (27B), served as LoRA (merge_and_unload into base for vLLM inference).
• LoRA config: rank=64, alpha=128, dropout=0.05, target_modules=all-linear (LLM + vision tower + aligner), freeze_llm/vit/aligner=false.
  • 9B: 205.06M trainable (2.13%) · 27B: 499M trainable (1.79%).
• Training: ms-swift swift sft, --num_train_epochs 15, --add_non_thinking_prefix true (Qwen3.5 non-thinking), max_length 4096, max_pixels 1048576 (native ~921K, no downscale), learning_rate 1e-4, cosine, optim adamw_torch_fused, bf16.
  • 9B: per-device batch 8 × 8 GPUs = eff 64 → 3225 steps.
  • 27B: per-device batch 16 × 8 GPUs = eff 128 → 1620 steps.
• Prompt: system ("You are a surgical assistant … Be precise and concise.") + <image> + question, non-thinking prefix <think>\n\n</think>\n\n appended at inference. (fo_definitions NOT injected in these runs.)

Eval (official focus.evaluation.Evaluator, vLLM inference + Qwen3.5-4B judge)

Micro accuracy (raw correct / total), object_recognition / aggregation groups.

Best: 9b-3225 (overall 69.40% / rec 77.11% / agg 60.07%). 9B > 27B by ~1pp. Weak spots: number (~51%, systematic under-count) and aggregation (~60%).

Usage

from peft import PeftModel
from transformers import AutoModelForImageTextToText, AutoProcessor

base = "Qwen/Qwen3.5-9B"  # or Qwen/Qwen3.8-27B
model = AutoModelForImageTextToText.from_pretrained(base, torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(model, "<repo>/9b-3225")
proc = AutoProcessor.from_pretrained(base)

For vLLM: merge_and_unload() the LoRA into the base first, then serve the merged weights (vLLM's PunicaWrapper does not support vision-tower LoRA).