ShaderAgent-MatScorer-Qwen3VLReranker-LoRA

ShaderAgent MatScorer — Qwen3-VL-Reranker LoRA

A LoRA adapter that turns Qwen/Qwen3-VL-Reranker-8B into MatScorer, the scorer used by ShaderAgent to rank procedural material renders against a reference image region.

• Rank: 8, alpha: 32, dropout: 0.05, target: all linear layers of the language-model part (q/v/k/gate/up/down/o_proj).
• Score: sigmoid((W_yes − W_no) · h_last) on the last hidden state — a continuous relevance in [0, 1].
• Input: a reference image + pixel-space bbox marking the target region, followed by the three canonical views of one candidate material (plane, bright_ball, dark_ball).

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

base = AutoModelForImageTextToText.from_pretrained(
    "Qwen/Qwen3-VL-Reranker-8B", device_map="cuda", torch_dtype="bfloat16"
)
model = PeftModel.from_pretrained(base, "yuanze1024/ShaderAgent-MatScorer-Qwen3VLReranker-LoRA")
model = model.merge_and_unload()  # optional

The exact prompt format, bbox <|box_start|> encoding and scoring head live in ShaderAgent's shader_agent/scoring/metrics.py (VLMScorer). Set:

image_metrics:
  model_path: Qwen/Qwen3-VL-Reranker-8B
  adapter_path: yuanze1024/ShaderAgent-MatScorer-Qwen3VLReranker-LoRA

Training data

Trained on the ShaderAgent-MatScorer-Data dataset: S1 synthetic pairs (retrieval-similarity labels) followed by S2 human forced-choice pairs. The listwise loss is InfoNCE over [positive, neg_1 … neg_N], so the negatives within one record are order-independent.

License

These weights are for research / non-commercial use.

The base model, Qwen/Qwen3-VL-Reranker-8B, is Apache-2.0 — but this adapter is not, so the repo is labeled license: other rather than inheriting the base model's badge.

The adapter was trained on ShaderAgent-MatScorer-Data, whose images are renders derived from several upstream collections with different terms: BlenderKit materials (not open source; renders only, no asset files redistributed), MatSynth (CC0 with a CC-BY subset), and Tencent Hunyuan 3D objects (community license, which excludes the EU, the UK and South Korea from its territory). The dataset's NOTICE gives the terms per directory.

These are trained weights, not a redistribution of any upstream asset, and no upstream licensor endorses them. If your use is commercial, clear the upstream terms yourself rather than relying on this repository.

Citation

Please cite the ShaderAgent paper if you use these weights.