Downloads · 30 days
11
10% of all-time downloads
ilia-dybal/Qwen-2.5-VL-3b-Instruct-4WA16-GPTQ-Culinary
Qwen-2.5-VL-3b-Instruct-4WA16-GPTQ-Culinary is a image-text-to-text model from ilia-dybal. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
The repository contains a finetuned version of Qwen-2.5-VL-3B-Instruct model. The model was finetuned on custom vision dataset specialized for culinary tasks: structured receipe and engredients extraction.
Downloads · 30 days
11
10% of all-time downloads
All-time downloads
112
Public
Parameters
3.8B
3.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.4 GB · 100%
How the weights are stored.
I322.8B · 74%
From the Hugging Face model README
The repository contains a finetuned version of Qwen-2.5-VL-3B-Instruct model. The model was finetuned on custom vision dataset specialized for culinary tasks: structured receipe and engredients extraction.
The model was initially finetuned in a A100 GPU with 4-bit QLoRA and post-quantized with llm-compressor using W6A16 scheme and GPTQ algorithm. The vision encoder retained untouched to preserce dynamic resolution capabilities.
Qwen2.5-VL-3B-Instruct).[More Information Needed]
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
Use the code below to get started with the model.
Launch the vLLM server from your terminal:
vllm serve ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary \
--trust-remote-code \
--max-model-len 4096 \
--limit-mm-per-prompt '{"image": 1}'
Query the server using the OpenAI Python Client:
import base64
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")
def encode_image(image_path):
with open(image_path, "rb") as f:
return f"data:image/jpeg;base64,{base64.b64encode(f.read()).decode('utf-8')}"
response = client.chat.completions.create(
model="ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary",
messages=[
{
"role": "user",
"content": [
{"type": "text", "text": "Analyze this dish and provide the recipe."},
{"type": "image_url", "image_url": {"url": encode_image("dish.jpg")}}
]
}
],
max_tokens=512
)
print(response.choices[0].message.content)
import torch
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
model_id = "ilia-dybal/Qwen-2.5-VL-3b-Instruct-4bit-LoRA-culinary"
processor = AutoProcessor.from_pretrained(model_id)
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
# Prepare multimodal input
messages = [
{
"role": "user",
"content": [
{"type": "image", "image": "[https://example.com/dish.jpg](https://example.com/dish.jpg)"},
{"type": "text", "text": "List the key ingredients in this photo."}
]
}
]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=None, return_tensors="pt").to("cuda")
output_ids = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output_ids[0], skip_special_tokens=True))
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).