Downloads · 30 days
19
10% of all-time downloads
Vedant3907/qwen2-vl-2b-finetuned-amazon-ml-challenge-2025
qwen2-vl-2b-finetuned-amazon-ml-challenge-2025 is a image-text-to-text model from Vedant3907. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
Qwen2-VL-2B-Instruct fine-tuned to predict a product's retail price in USD from its catalog text (item name, bullet points, value/unit, pack quantity) and product image.
Downloads · 30 days
19
10% of all-time downloads
All-time downloads
199
Public
Parameters
2.2B
4.4 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors4.4 GB · 100%
From the Hugging Face model README
Qwen2-VL-2B-Instruct fine-tuned to predict a product's retail price in USD from its catalog text (item name, bullet points, value/unit, pack quantity) and product image.
Trained on Amazon ML Challenge 2025 with Unsloth QLoRA on a single Kaggle GPU, then merged back into the base weights — this repo is the full self-contained 16-bit model (~4.4 GB), no adapter loading required.
Given a product's catalog content (text) and image, predict the retail price in USD.
| Base model | Qwen/Qwen2-VL-2B-Instruct (Apache-2.0) — loaded as unsloth/Qwen2-VL-2B-Instruct-bnb-4bit for training |
| Method | QLoRA via Unsloth (4-bit NF4 + double quant), then merged to 16-bit fp |
| LoRA | r = 16, α = 16, dropout = 0, no bias |
| Target modules | language + attention + MLP layers (vision tower frozen for speed) |
| Trainable params | ~28.9M (1.3% of total) |
| Training data | ~74,198 catalog/image/price triples (after dropping rows with no image/price) |
| Validation | 800-row stratified split (10 price bins) |
| Image size | 336 × 336 (144 vision tokens) |
| Max sequence length | 1024 |
| Batch | 4 per device × grad-accum 4 = effective 16 |
| Optimizer | adamw_8bit |
| Learning rate | 2e-4, cosine schedule, 3% warmup |
| Weight decay | 0.01 |
| Precision | fp16 |
| Epochs | 1 (~4,638 steps total; merged from step 4000, val loss 0.908) |
| Seed | 3407 |
| Hardware | Kaggle T4 (single GPU, ~16 GB VRAM) |
Evaluation ran every 600 steps. The model was merged from the step-4000 checkpoint; the last logged eval before the session ended was at step 3600.
| Step | Training loss | Validation loss |
|---|---|---|
| 600 | 1.0996 | 1.0725 |
| 1200 | 0.9814 | 1.0159 |
| 1800 | 0.9633 | 0.9736 |
| 2400 | 0.9842 | 0.9437 |
| 3000 | 0.9774 | 0.9209 |
| 3600 | 0.9501 | 0.9079 |
You are an expert e-commerce pricing assistant. Given a product's catalog text and image,
predict its retail price in US dollars. Do NOT copy numbers from weights, volumes,
quantities, or pack sizes (like '12 Ounce', '11.25 oz', 'Pack of 6') as the price.
Consider product type, brand and image cues. Reply with ONLY the price as a number, e.g. 12.99.
<catalog_content (truncated to 1200 chars)>
[image]
Assistant response is just the numeric price, e.g. 12.99.
enable_input_require_grads + non-reentrant checkpointing automatically.import torch
from PIL import Image
from transformers import Qwen2VLForConditionalGeneration, AutoProcessor
REPO = "Vedant3907/qwen2-vl-2b-amazon-pricing-merged"
processor = AutoProcessor.from_pretrained(REPO, min_pixels=128*28*28, max_pixels=256*28*28)
model = Qwen2VLForConditionalGeneration.from_pretrained(
REPO, torch_dtype=torch.float16, device_map="auto")
model.eval()
INSTRUCTION = (
"You are an expert e-commerce pricing assistant. Given a product's catalog text and image, "
"predict its retail price in US dollars. "
"Do NOT copy numbers from weights, volumes, quantities, or pack sizes "
"(like '12 Ounce', '11.25 oz', 'Pack of 6') as the price. "
"Consider product type, brand and image cues. Reply with ONLY the price as a number, e.g. 12.99."
)
@torch.no_grad()
def predict(catalog_text: str, image_path: str) -> str:
img = Image.open(image_path).convert("RGB")
msgs = [{"role": "user", "content": [
{"type": "text", "text": f"{INSTRUCTION}\n\n{catalog_text[:1200]}"},
{"type": "image", "image": img},
]}]
text = processor.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inp = processor(text=[text], images=[img], return_tensors="pt").to(model.device)
out = model.generate(**inp, max_new_tokens=8, do_sample=False)
gen = out[:, inp["input_ids"].shape[1]:]
return processor.batch_decode(gen, skip_special_tokens=True)[0].strip()
print(predict(
"Brand: Acme. Wireless Bluetooth Earbuds with 24h battery, IPX5 water resistant.",
"earbuds.jpg",
))
# -> "29.99"
Same recipe used to produce this model. Run on a Kaggle T4 (or any 16 GB GPU) with internet ON.
# pip install -q unsloth transformers==4.55.4 --no-deps trl==0.22.2
import os
os.environ["TOKENIZERS_PARALLELISM"] = "false"
import pandas as pd, torch
from PIL import Image
from torch.utils.data import Dataset as TorchDS
from unsloth import FastVisionModel
from unsloth.trainer import UnslothVisionDataCollator
from trl import SFTTrainer, SFTConfig
SEED = 3407
MODEL_ID = "unsloth/Qwen2-VL-2B-Instruct-bnb-4bit"
INSTRUCTION = (
"You are an expert e-commerce pricing assistant. Given a product's catalog text and image, "
"predict its retail price in US dollars. "
"Do NOT copy numbers from weights, volumes, quantities, or pack sizes "
"(like '12 Ounce', '11.25 oz', 'Pack of 6') as the price. "
"Consider product type, brand and image cues. Reply with ONLY the price as a number, e.g. 12.99."
)
# --- load data ---
df = pd.read_csv("train.csv").dropna(subset=["catalog_content", "price"])
df = df[df["price"] > 0].reset_index(drop=True)
df["image_path"] = df["sample_id"].astype(str).map(lambda s: f"images/{s}.jpg")
df = df.sample(frac=1, random_state=SEED).reset_index(drop=True)
df["_bin"] = pd.qcut(df["price"], q=10, labels=False, duplicates="drop")
val_df = df.groupby("_bin", group_keys=False).apply(lambda g: g.sample(80, random_state=SEED))
train_df = df.drop(val_df.index).reset_index(drop=True)
# --- model + LoRA ---
model, tokenizer = FastVisionModel.from_pretrained(
MODEL_ID, load_in_4bit=True, use_gradient_checkpointing="unsloth")
model = FastVisionModel.get_peft_model(
model,
finetune_vision_layers = False, # vision frozen for speed
finetune_language_layers = True,
finetune_attention_modules = True,
finetune_mlp_modules = True,
r = 16, lora_alpha = 16, lora_dropout = 0, bias = "none",
random_state = SEED,
)
# --- dataset ---
def to_messages(row):
return [
{"role": "user", "content": [
{"type": "text", "text": f"{INSTRUCTION}\n\n{str(row['catalog_content'])[:1200]}"},
{"type": "image", "image": Image.open(row["image_path"]).convert("RGB")},
]},
{"role": "assistant", "content": [{"type": "text", "text": f"{float(row['price']):.2f}"}]},
]
class PriceVision(TorchDS):
def __init__(self, frame): self.frame = frame.reset_index(drop=True)
def __len__(self): return len(self.frame)
def __getitem__(self, i): return {"messages": to_messages(self.frame.iloc[i].to_dict())}
FastVisionModel.for_training(model)
# --- trainer ---
trainer = SFTTrainer(
model = model, tokenizer = tokenizer,
data_collator = UnslothVisionDataCollator(model, tokenizer, resize=(336, 336)),
train_dataset = PriceVision(train_df), eval_dataset = PriceVision(val_df),
args = SFTConfig(
per_device_train_batch_size = 4, gradient_accumulation_steps = 4,
num_train_epochs = 1, learning_rate = 2e-4,
warmup_ratio = 0.03, lr_scheduler_type = "cosine", weight_decay = 0.01,
optim = "adamw_8bit", fp16 = True, bf16 = False,
logging_steps = 25, eval_strategy = "steps", eval_steps = 600,
save_strategy = "steps", save_steps = 200, save_total_limit = 2,
seed = SEED, output_dir = "qwen2vl_unsloth_lora", report_to = "none",
remove_unused_columns = False, dataset_text_field = "",
dataset_kwargs = {"skip_prepare_dataset": True}, max_length = 1024,
),
)
trainer.train()
# --- merge + push the full model to the Hub ---
model.push_to_hub_merged(
"Vedant3907/qwen2-vl-2b-amazon-pricing-merged",
tokenizer,
save_method="merged_16bit",
token="<your-hf-write-token>",
)
To resume from a checkpoint (e.g. after a session timeout), pass the checkpoint path:
trainer.train(resume_from_checkpoint="qwen2vl_unsloth_lora/checkpoint-4000")
Use the same SEED so the data order is identical.
-?\d+(?:\.\d+)?) and fall back to a sensible median if parsing fails.Apache-2.0 (inherited from Qwen/Qwen2-VL-2B-Instruct).
@misc{vedant2026qwen2vl-amazon-pricing,
author = {Vedant Rajpurohit},
title = {Qwen2-VL-2B Fine-tuned for Amazon ML Challenge 2025 (Smart Product Pricing)},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/Vedant3907/qwen2-vl-2b-amazon-pricing-merged}},
}