qwen25vl-container-defects-lora

Qwen2.5-VL-3B Container Defect Detector (LoRA)

LoRA adapter for Qwen2.5-VL-3B-Instruct, fine-tuned to detect and classify visible defects on shipping containers (dents, rust, scratches, holes, frame damage) directly from a photo. Built for ContainerHub.

Model Details

Model Description

Given a photo of a shipping container, the model returns a comma-separated list of the visible defect labels (e.g. Dent, Rusty, Scratch). It is a LoRA adapter — load it on top of the base vision-language model, not standalone.

• Developed by: abejarano (ContainerHub)
• Model type: Vision-language model, LoRA adapter (PEFT)
• Language(s): English
• License: qwen-research (inherited from the base model)
• Finetuned from model: unsloth/Qwen2.5-VL-3B-Instruct-bnb-4bit (base: Qwen/Qwen2.5-VL-3B-Instruct)

Model Sources

• Training data: abejarano/container-defects-qwen

Uses

Direct Use

Multi-label defect classification of shipping container photos, prompted as: "What defect on the shipping container?" → free-text answer listing defect labels.

Out-of-Scope Use

Not intended for defect severity grading, safety certification, or any decision where a missed or false defect classification has safety/financial consequences without human review. Not validated on container types, lighting conditions, or camera angles outside the training distribution.

Bias, Risks, and Limitations

• Trained on 1,016 images total, with no held-out validation split — reported loss reflects training-set fit, not generalization to unseen containers.
• Early experiments (3 epochs, LoRA r=16) showed the model could hallucinate near-miss labels (e.g. "Dentation" instead of "Dent") even under greedy decoding. This version (6 epochs, LoRA r=32) fixes that on the training examples checked, but a proper eval split is still recommended before production use.
• Label vocabulary is fixed to the 5 classes present in the source dataset (Dent, Rusty, Scratch, Hole, Deframe); it will not recognize defect types outside this set.

Recommendations

Evaluate on a held-out set of container images before relying on this model for automated decisions. Keep a human in the loop for edge cases.

How to Get Started with the Model

from unsloth import FastVisionModel
from PIL import Image

model, tokenizer = FastVisionModel.from_pretrained(
    "abejarano/qwen25vl-container-defects-lora",
    load_in_4bit=True,
)
FastVisionModel.for_inference(model)

image = Image.open("container.jpg").convert("RGB")
messages = [
    {"role": "user", "content": [
        {"type": "image", "image": image},
        {"type": "text", "text": "What defect on the shipping container?"},
    ]}
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt",
).to("cuda")

output = model.generate(**inputs, max_new_tokens=150, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Training Details

Training Data

abejarano/container-defects-qwen — 1,016 container images converted to Qwen conversational format, derived from howell0123/shipping_container. Labels: Dent, Rusty, Scratch, Hole, Deframe (multi-label, comma-separated).

Training Procedure

Fine-tuned with Unsloth + LoRA on a single RTX 4090 (RunPod, unsloth/unsloth:latest template).

Preprocessing

Images decoded to PIL from the dataset's embedded bytes at load time; conversation format {"role": ..., "content": [{"type": "image"/"text", ...}]} fed directly to UnslothVisionDataCollator.

Training Hyperparameters

• LoRA rank (r): 32
• LoRA alpha: 32
• Target modules: all-linear (vision + language + attention + MLP layers)
• Trainable params: 82,169,856 (2.14% of 3.8B)
• Epochs: 6
• Batch size: 2 per device, gradient accumulation 4 (effective batch size 8)
• Learning rate: 2e-4, cosine schedule, 3% warmup
• Optimizer: adamw_8bit
• Training regime: bf16 mixed precision

Speeds, Sizes, Times

• Train runtime: 1942.7s (~32 min)
• Train samples/sec: 3.14
• Final train loss: 0.128 (per-step loss ~0.0002–0.001 by the last epoch)
• Adapter size: ~329MB (adapter_model.safetensors)

Evaluation

Informal validation only: predictions (greedy decoding) were spot-checked against ground-truth labels on training examples. No dedicated held-out evaluation split exists yet — see Limitations above.

Environmental Impact

• Hardware Type: NVIDIA RTX 4090
• Hours used: ~0.85h GPU time across training runs
• Cloud Provider: RunPod
• Compute Region: RunPod on-demand pod (region not pinned)

Technical Specifications

Model Architecture and Objective

Qwen2.5-VL-3B-Instruct (vision-language transformer) with LoRA adapters, trained via supervised fine-tuning (SFT) framed as multi-label defect classification through free-text generation.

Compute Infrastructure

• Hardware: 1x RTX 4090 24GB, RunPod pod
• Software: unsloth 2026.5.9, transformers 4.57.6, trl (SFTTrainer), peft 0.18.1, torch 2.10.0+cu128

Framework versions

• PEFT 0.18.1