smol-product-description-vlm-eval

Model Card for RahulKate-173/smol-product-description-vlm-eval

<!-- Provide a quick summary of what the model is/does. -->

This model is a fine-tuned Vision-Language Model (VLM) based on SmolVLM, optimized for generating accurate, structured, and descriptive product descriptions from e-commerce product images and multimodal inputs.

Model Details

Model Description

• Developed by: Rahul Ravindra Kate
• Model type: Vision-Language Model (Image-Text-to-Text)
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model [optional]: HuggingFaceTB/SmolVLM-256M-Instruct

Model Sources [optional]

• Repository: GitHub / Hugging Face Repository

Uses

Direct Use

The model is intended to take product images as input and generate rich, SEO-friendly, and informative product descriptions suitable for e-commerce listings, cataloging, and automated tagging.

Downstream Use [optional]

Can be integrated into automated inventory management pipelines, e-commerce backend services, or multi-agent catalog generation systems.

Out-of-Scope Use

• Medical, legal, or high-stakes safety evaluations based purely on product packaging.
• General-purpose dense image captioning outside of commercial products.

Bias, Risks, and Limitations

• Hallucination: Like all lightweight VLMs, the model may occasionally hallucinate small product specifications (such as exact material composition, brand names not visible, or precise dimensions) if not clearly readable from the image.
• Bias: The training distribution primarily reflects standard commercial product photography; performance may degrade on blurry images, low-light conditions, or non-standard angles.

How to Get Started with the Model

Use the code below to get started with the model using the 🤗 transformers library.

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForVision2Seq

device = "cuda" if torch.cuda.is_available() else "cpu"

model_id = "RahulKate-173/smol-product-description-vlm-eval"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForVision2Seq.from_pretrained(
    model_id, 
    torch_dtype=torch.bfloat16 if device == "cuda" else torch.float32
).to(device)

# Load an example product image
image = Image.open("path_to_product_image.jpg")

# Prepare prompt
prompt = "Provide a detailed product description for the item shown in the image."
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image"},
            {"type": "text", "text": prompt}
        ]
    }
]

text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(device)

# Generate output
generated_ids = model.generate(**inputs, max_new_tokens=256)
generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

print(generated_text)