Downloads · 30 days
6
10% of all-time downloads
BDivyesh/boomi-stage3-vlm-cubicasa-research
boomi-stage3-vlm-cubicasa-research is a image-text-to-text model from BDivyesh. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as cc-by-nc-sa-4.0.
⚠️ NON-COMMERCIAL / RESEARCH-ONLY. This LoRA adapter was fine-tuned on CubiCasa5K (CC-BY-NC-SA 4.0). It is released under CC-BY-NC-SA 4.0 and must not be used commercially. It is not part of BOOMI's license-clean ship…
Downloads · 30 days
6
10% of all-time downloads
All-time downloads
60
Public
Repo size
160 MB
Likes
0
Public
Click a slice to open those files.
.safetensors149 MB · 90%
From the Hugging Face model README
⚠️ NON-COMMERCIAL / RESEARCH-ONLY. This LoRA adapter was fine-tuned on CubiCasa5K (CC-BY-NC-SA 4.0). It is released under CC-BY-NC-SA 4.0 and must not be used commercially. It is not part of BOOMI's license-clean shipped model line (those are the
BDivyesh/boomi-stage-a-*text→SPEC models, trained only on permissive data). Use it for research and benchmarking only, with attribution to CubiCasa5K.
A LoRA adapter for Qwen/Qwen2.5-VL-3B-Instruct (Apache-2.0 base) that reads a
residential floor-plan image and emits a structured room program as JSON:
{"room_program": {"BED ROOM": 3, "BATH": 2, "KITCHEN": 1, "LIVING": 1, "BALCONY": 2, "FOYER": 1, "STORE": 1, "LOBBY": 1}, "bhk": "3BHK"}
This is the raster modality of BOOMI's reverse pipeline (arbitrary drawing → structured SPEC), for scanned/photo plans where vector geometry and in-drawing text are unavailable. The SPEC it produces is intended to seed a downstream constraint solver — it is an assistive proposer a human reviews and corrects, not an autonomous extractor.
Scored vs the CubiCasa SVG room-class annotations (independent of the model's pixels), with room-type names canonicalised to a shared label space (synonyms folded on both sides).
| metric | base (zero-shot) | this adapter (fine-tuned) | lift |
|---|---|---|---|
| program multiset Jaccard | 0.360 | 0.618 | +0.259 |
| room-type presence recall | 0.622 | 0.869 | +0.248 |
| bedroom-count (BHK) accuracy | 0.339 | 0.556 | +0.217 |
| valid JSON | 0.993 | 1.000 | — |
| exact whole-program match | 0.000 | 0.034 | — |
Read this honestly: fine-tuning materially improves which room types are present and the rough bedroom count, but exact whole-program match is only ~3.4 % — counts of balconies, baths and stores routinely drift by ±1, and the model leans on learned program priors (a few templates cover a large share of outputs). It is useful as a draft generator, not a source of truth.
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from peft import PeftModel
from PIL import Image
BASE = "Qwen/Qwen2.5-VL-3B-Instruct"
ADAPTER = "BDivyesh/boomi-stage3-vlm-cubicasa-research"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
BASE, torch_dtype=torch.bfloat16, use_safetensors=True).to("cuda")
model = PeftModel.from_pretrained(model, ADAPTER).to("cuda").eval()
proc = AutoProcessor.from_pretrained(BASE, max_pixels=1280*28*28)
PROMPT = ('You are an expert architect reading a residential floor-plan image. '
'Identify every distinct room and return ONLY JSON: '
'{"room_program":{"<TYPE>":<count>,...},"bhk":"<n>BHK"}. '
'bhk = number of bedrooms. Count each room instance. Return only the JSON.')
img = Image.open("plan.png").convert("RGB")
msgs = [{"role": "user", "content": [{"type": "image", "image": "plan.png"},
{"type": "text", "text": PROMPT}]}]
text = proc.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
inp = proc(text=[text], images=[img], return_tensors="pt").to("cuda")
out = model.generate(**inp, max_new_tokens=512, do_sample=False)
print(proc.batch_decode(out[:, inp.input_ids.shape[1]:], skip_special_tokens=True)[0])
Qwen/Qwen2.5-VL-3B-Instruct (Apache-2.0).MH, ET). Indian / other-region transfer is unvalidated.Qwen/Qwen2.5-VL-3B-Instruct: Apache-2.0 (its own license applies).