Downloads · 30 days
7
18% of all-time downloads
Mexico-Oxford-TEAM/frame-qwen3vl-8b-r42-ep2
frame-qwen3vl-8b-r42-ep2 is a image-text-to-text model from Mexico-Oxford-TEAM. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as cc-by-nc-sa-4.0.
Model B of the checkpoint pair that scored 0.58128 on the official FRAME leaderboard. Designed to be run alongside r42-ep4, not on its own.
Downloads · 30 days
7
18% of all-time downloads
All-time downloads
40
Public
Parameters
8.8B
17.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors17.6 GB · 100%
From the Hugging Face model README
Model B of the checkpoint pair that scored 0.58128 on the official FRAME leaderboard. Designed to be run alongside r42-ep4, not on its own.
Fine-tuned Qwen3-VL-8B-Instruct for the
ORENA SAVE FOCUS Challenge — FRAME track
(MICCAI 2026), by team Mexico-Oxford_TEAM.
📦 Code, training configs, every experiment and every negative result:
RodMed0709/ORENA-Challenge-MEXICO
Our submitted system runs two checkpoints, not one. This repository is model B; its
partner is frame-qwen3vl-8b-r42-ep4 (model A).
Both are epochs of the same rung 42 training run — same data, same recipe, different epoch.
The rule. Model A answers. If A's answer parses entirely as legal foreign-object class names, model B is asked the same question, and if B returns a strictly shorter list, B's answer wins; ties and anything that is not a class list keep A. Numbers, yes/no and multiple-choice answers never parse as class lists, so they fall through untouched.
ans_a = ask(model_a, question, frame)
ans_b = ask(model_b, question, frame) # only needed when ans_a is a class list
final = ans_b if is_class_list(ans_a) and is_class_list(ans_b) \
and len(set_of(ans_b)) < len(set_of(ans_a)) else ans_a
Why it works, measured on fo_class questions:
| model A alone | with the pair | |
|---|---|---|
| out-of-centre probe (another hospital) | 0.3618 | 0.3838 (+0.0221, 15/15 leave-one-video-out folds) |
| in-domain control | 0.8408 | 0.8367 (−0.0041, within noise) |
The mechanism is not "two heads are better". This checkpoint's dominant error is naming a class that is not in the frame, and a differently-trained sibling disagrees about which extra class often enough to delete some of them.
🔴 The direction is the whole lever. Taking the union of the two answers instead of the shorter list costs −0.1562. Intersect, never combine.
Either checkpoint also works alone — that is what our submission 03 did, and it scored 0.5809.
This is model B, so it has no solo score. The pair — r42-ep4 as A, this as B — scored
0.58128, against official baselines of 0.5189 (fine-tuned) and 0.3883 (proprietary).
| root | the full merged model — load it directly, no extra step |
adapter/ | the LoRA adapter alone (~100 MB), if you would rather apply it to the base yourself |
The root holds the standard Hugging Face file set for this architecture — the same one
Qwen/Qwen3-VL-8B-Instruct ships: config.json (architecture and dtype), the tokenizer files
(tokenizer.json, vocab.json, merges.txt, added_tokens.json, special_tokens_map.json,
tokenizer_config.json), chat_template.jinja (the prompt format — without it the model is
prompted wrong and answers badly), preprocessor_config.json and
video_preprocessor_config.json (how images and video frames are resized and normalised),
generation_config.json, and model.safetensors.index.json (which tensor lives in which shard).
from transformers import AutoProcessor, AutoModelForImageTextToText
model = AutoModelForImageTextToText.from_pretrained(
"Mexico-Oxford-TEAM/frame-qwen3vl-8b-r42-ep2", dtype="bfloat16", device_map="auto")
processor = AutoProcessor.from_pretrained("Mexico-Oxford-TEAM/frame-qwen3vl-8b-r42-ep2")
These are the versions this checkpoint was merged and evaluated on — the exact set our submission container ran with:
transformers==4.57.6
torch==2.5.1 # CUDA 12.4 build
qwen-vl-utils==0.0.14
accelerate>=1.0.0,<2.0
numpy>=1.23.0,<2.0 # numpy 2.x is untested against this torch build
pillow>=9.0,<14.0
⚠️ Stay on the
transformers4.57 line. The config here is written in the 4.57 layout, and 5.x writes an incompatible one for this architecture. Mixing the two produces a load-time failure, not a silent degradation, so you will know immediately.
ms-swift 4.4, tuner_type: lora. Every value is
read from adapter/args.json in this repository, not from notes:
| base model | Qwen/Qwen3-VL-8B-Instruct (Apache-2.0) |
| LoRA rank / alpha / dropout | 8 / 32 / 0.1 |
| target modules | all-linear plus the vision merger and all three deepstack mergers |
| vision tower | not frozen (freeze_vit: false, freeze_aligner: false) |
| learning rate | 2e-4, cosine, warmup 0.03, weight decay 0.1 |
| batch | 1 x grad-accum 16 |
| epochs | 5 total; this is epoch 2 |
| precision / seed | bfloat16 / 42 |
Research artifact from a benchmark challenge on laparoscopic surgical video. It answers short questions about foreign objects — clips, drains, specimen bags, needles, threads — in a frame.
🔴 Not a medical device. Not for clinical decision-making.
Known weakness, measured rather than assumed: it cannot enumerate reliably past about two objects. On external laparoscopic frames with large metallic instruments, accuracy falls to 0.04 at three objects. That is a perception limit, not a formatting one — it reproduces across two unrelated answer formats and two different datasets.
Trained only on the challenge's own corpus (ORENA SAVE FOCUS FRAME:
heico-focus-vqa +
lapchole-focus-vqa), built on
the Heidelberg Colorectal dataset
(doi:10.1038/s41597-021-00882-2). No external
surgical corpus is in this checkpoint — several were tried and each closed as a measured negative.
CC BY-NC-SA 4.0. The backbone is Apache-2.0, but the fine-tuning data is CC BY-NC-SA and the stricter term governs the derivative. NonCommercial, ShareAlike, attribution required.
| Code and experiments | RodMed0709/ORENA-Challenge-MEXICO |
| Challenge | FRAME track |
| Training data | heico-focus-vqa · lapchole-focus-vqa — 🔒 access must be requested from the organizers |
Mexico-Oxford_TEAM
| GitHub | Hugging Face |
|---|---|
Leodna | LeOkna |
RodMed0709 | RodMed0709 |
YingyuYyy | TODO |
📌 TODO — team members. Names, affiliations, ORCIDs and roles — each member fills in their own. (placeholder: to be completed by the team)