Downloads · 30 days
14
5% of all-time downloads
Niraya666/Qwen3-4B-wmvlm-260204
Qwen3-4B-wmvlm-260204 is a image-text-to-text model from Niraya666. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This repository contains a supervised fine-tuned vision-language model for wafer map defect understanding.
Downloads · 30 days
14
5% of all-time downloads
All-time downloads
289
Public
Parameters
4.4B
8.9 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors8.9 GB · 100%
From the Hugging Face model README
This repository contains a supervised fine-tuned vision-language model for wafer map defect understanding.
It is part of WaferSAGE, a framework for wafer defect visual question answering via synthetic data generation, rubric-guided evaluation, and domain-specific post-training.
Historical note: this model was originally released under the
WMVLMnaming convention. It is now part of the WaferSAGE model family.
WaferSAGE-SFT is fine-tuned from Qwen3-VL on synthetic wafer map VQA data. The model is designed to answer natural language questions about wafer map images, including defect type identification, spatial distribution analysis, morphology description, and root-cause hypothesis generation.
| Item | Description |
|---|---|
| Project | WaferSAGE |
| Model Type | Vision-Language Model |
| Base Model | Qwen3-VL Instruct |
| Fine-tuning Method | LoRA-SFT |
| Task | Image-text-to-text / Visual Question Answering |
| Domain | Semiconductor wafer map defect analysis |
| Language | English |
The model can answer questions such as:
<image>
What type of defect pattern is visible on this wafer map?
<image>
Where are the defects located on the wafer?
<image>
Describe the morphology and spatial distribution of this wafer map defect.
<image>
What are the possible root-cause hypotheses for this defect pattern?
This model was trained on WaferSAGE wafermap VQA data, generated through a multi-stage synthetic data pipeline:
Wafer map images + labels
↓
Clustering-based cleaning and sampling
↓
VLM-generated defect descriptions
↓
Structured rubric extraction
↓
VQA synthesis
↓
LoRA supervised fine-tuning
The VQA data covers:
The SFT models were trained with LoRA adaptation on Qwen3-VL.
Typical configuration:
| Hyperparameter | Value |
|---|---|
| Fine-tuning method | LoRA |
| LoRA rank | 16 |
| LoRA alpha | 16 |
| LoRA dropout | 0 |
| Optimizer | AdamW 8-bit |
| Learning rate | 2e-4 |
| Scheduler | Linear |
| Epochs | 1 |
| Max context length | 2048 |
| Vision layers | Fine-tuned with LoRA |
| Language layers | Fine-tuned with LoRA |
| Attention modules | Fine-tuned with LoRA |
| MLP modules | Fine-tuned with LoRA |
from transformers import pipeline
pipe = pipeline(
"image-text-to-text",
model="Niraya666/Qwen3-4B-wmvlm-260204"
)
messages = [
{
"role": "user",
"content": [
{"type": "image", "url": "path_or_url_to_wafermap.png"},
{"type": "text", "text": "What type of defect pattern is visible on this wafer map?"}
]
}
]
output = pipe(text=messages, max_new_tokens=256)
print(output)
vllm serve "Niraya666/Qwen3-4B-wmvlm-260204"
Then call the OpenAI-compatible endpoint:
curl -X POST "http://localhost:8000/v1/chat/completions" \
-H "Content-Type: application/json" \
--data '{
"model": "Niraya666/Qwen3-4B-wmvlm-260204",
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Describe the wafer map defect pattern."
},
{
"type": "image_url",
"image_url": {
"url": "path_or_url_to_wafermap.png"
}
}
]
}
],
"max_tokens": 256
}'
The model was evaluated using WaferSAGE's rubric-based wafermap VQA benchmark.
Evaluation dimensions include:
The evaluation combines:
The SFT model primarily improves domain-specific terminology, response format, and wafermap-specific visual reasoning compared with the base VLM. For stronger performance, see the WaferSAGE RL models trained with GSPO and rubric-aligned rewards.
This model is a domain-adapted VLM for wafer map understanding, but it has important limitations:
For production semiconductor engineering use, model outputs should be reviewed by qualified engineers and combined with process context.
Recommended:
Not recommended:
Niraya666/wafermap-vqa-2602Niraya666/wafermap-vqa-with-rubrics-2602, Niraya666/wafermap-vqa-with-rubrics-2602_v2Niraya666/Qwen3-VL-4B-Instruct-WMVLM-RL-0213If you use this model, please cite:
@misc{xu2026wafersage,
title = {WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning},
author = {Ke Xu and Zhongyuan Lian},
year = {2026},
eprint = {2604.27629},
archivePrefix= {arXiv},
primaryClass = {cs.AI}
}