Downloads · 30 days
53
31% of all-time downloads
omniparser-icon-caption-mlx
PlusMinus1/omniparser-icon-caption-mlx
omniparser-icon-caption-mlx is a image-text-to-text model from PlusMinus1. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for mlx. The card lists the license as mit.
MLX (bfloat16) conversion of microsoft/OmniParser-v2.0's iconcaption — a Florence-2 fine-tuned on UI elements, for captioning interactive icons in screenshots. Runs on Apple Silicon via mlxvlm with no PyTorch.
All-time downloads
172
Public
Parameters
271M
542 MB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors542 MB · 99%
At a glance
- Task
- Image-Text-to-Text
- Library
- mlx
- License
- mit
- Model type
- florence2
- Access
- Public
- Created
- Jun 19, 2026
- Updated
- Jun 19, 2026
- SHA
- 6c63a7da
Try a prompt
Base models
Model card
From the Hugging Face model README
OmniParser icon_caption — MLX
MLX (bfloat16) conversion of microsoft/OmniParser-v2.0's icon_caption — a Florence-2 fine-tuned on UI elements, for captioning interactive icons in screenshots. Runs on Apple Silicon via mlx_vlm with no PyTorch.
License: MIT (© Microsoft Corporation) — see LICENSE. This repo redistributes the original MIT-licensed weights converted to MLX format.
Usage
Needs a small no-torch patch on transformers 5.x (register florence2_language, route the image processor to CLIPImageProcessorPil). See the conversion recipe at the bottom.
# apply the florence2 no-torch patch first (see recipe), then:
from mlx_vlm import load, generate
model, processor = load("PlusMinus1/omniparser-icon-caption-mlx")
out = generate(model, processor, "<CAPTION>", image=["icon_crop.png"], max_tokens=20)
Provenance
- Base: microsoft/Florence-2-base (MIT)
- Fine-tune: microsoft/OmniParser-v2.0
icon_caption(MIT) - Conversion:
mlx_vlm.convert(bfloat16) + a transformers-5.x no-torch compatibility patch.
- Task
- Image-Text-to-Text
- Library
- mlx
- Type
- florence2
- License
- mit
- Created
- Jun 19, 2026
- Updated
- Jun 19, 2026