slugvision-2.2b

slugvision-2.2b

A 2.2B-parameter vision-language model that turns an uploaded image (plus an optional article title) into a 3–5 word kebab-case permalink slug. It is the quality-leader sibling of slugvision-500m: on untitled photos the blind LLM-judge gap to the 26B-class teacher drops from ~1.1 (500M) to 0.63, at ~10x the CPU latency (~16s vs 1.6s cold per image on 6 desktop threads).

LoRA fine-tune (r16, merged) of SmolVLM2-2.2B-Instruct, distilled from a Gemma teacher on ~31k image→slug pairs spanning five slices of an editorial-image distribution: everyday photos, conceptual editorial illustrations, product shots, clinical stock imagery, and web screenshots. Training data: kelnei/slugvision.

Pick this model if untitled-photo slug quality matters more than latency; pick the 500M if you serve on CPU and most uploads come with an article title (titled modes are near teacher parity at both sizes). The full pipeline (corpus generation, training, eval, export) is at github.com/kelnei/slugvision.

Input contract

Trained on exactly two prompt templates — use them verbatim.

Without a title (slug describes visible content):

Create a URL slug for this image: 3 to 5 lowercase words joined by hyphens. Reply with the slug only.

With an article title (slug reflects the article topic; image steers emphasis):

This image appears in an article titled: "{title}"
Create a URL slug for the image in this article's context: 3 to 5 lowercase words joined by hyphens. Reply with the slug only.

Images were preprocessed with size={"longest_edge": 1024} during training; match it at inference (the bundled GGUF mmproj embeds this cap).

Usage

llama.cpp

llama-mtmd-cli -m slugvision-2.2b-q8_0.gguf --mmproj mmproj-f16.gguf \
  --image photo.jpg -n 32 --temp 0 \
  -p 'Create a URL slug for this image: 3 to 5 lowercase words joined by hyphens. Reply with the slug only.'

Transformers

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

processor = AutoProcessor.from_pretrained("kelnei/slugvision-2.2b", size={"longest_edge": 1024})
model = AutoModelForImageTextToText.from_pretrained(
    "kelnei/slugvision-2.2b", torch_dtype=torch.bfloat16, device_map="auto"
)

messages = [{"role": "user", "content": [
    {"type": "image", "url": "photo.jpg"},
    {"type": "text", "text": "Create a URL slug for this image: 3 to 5 lowercase words joined by hyphens. Reply with the slug only."},
]}]
inputs = processor.apply_chat_template(
    messages, add_generation_prompt=True, tokenize=True,
    return_dict=True, return_tensors="pt"
).to(model.device, dtype=torch.bfloat16)
out = model.generate(**inputs, max_new_tokens=32, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Files

Evaluation

Blind pairwise LLM-judge against the teacher on a frozen 1,000-image holdout (rubric 1–5 per slug; the judge never knows which is which). Token-F1 is overlap with the teacher's slug.

Format validity ~99–100% across slices. In the 500M/2.2B capacity ladder, model size was the only lever that moved untitled-photo quality — 4x photo data, extra slices, and LoRA r64 all left the 500M's gap at ~1.0; this model cut it to 0.63 in one step.

CPU latency (llama.cpp, 6 threads, full cold start per image): ~16s at Q8_0 or Q4_K_M — image encode dominates (the 2.2B vision tower cuts ~9 crops per image at the 1024px cap). On any modest GPU it is sub-second.

Training

LoRA r16 on all linear layers, 1 epoch over 30,739 pairs, batch 8 with grad-accum 2, bf16, images capped at 1024px longest edge. Labels authored by a Gemma-family teacher (26B-class, locally hosted); titled-mode labels were generated with the source article headline in context.

License and provenance

• Base model: SmolVLM2-2.2B-Instruct, Apache-2.0.
• The fine-tuning labels are outputs of a Gemma-family model, so this distilled model is distributed under the Gemma Terms of Use with its prohibited use policy; hence the gemma license tag.
• Photo-slice training images came from COCO train2017 and are not redistributed here or in the dataset repo; all other training images are locally generated (FLUX.1-schnell renders, self-authored HTML screenshots) and ship with the dataset.

Limitations

• English slugs only.
• Tuned for editorial-image distributions (article cards); out-of-domain images (documents, memes, dense text) are untested.
• The slug is not guaranteed unique — collision handling belongs in your permalink generator.