AlphaRoute-VL-0.8B

AlphaRoute-VL-0.8B (Preview)

AlphaRoute-VL-0.8B is a sub-1B parameter Vision-Language Small Language Model designed for instruction-conditioned multimodal semantic routing, visual entity extraction, and structured JSON output.

AlphaRoute learns the meta-task of routing: it does not rely on static, hardcoded labels. Given an arbitrary task instruction, a dynamic ontology (with descriptions), a target JSON schema, and a visual input (or text query), AlphaRoute-VL parses the evidence and outputs structured JSON conforming to the requested schema.

---

⚠️ Important Architectural & Training Disclosure

Honest Disclosure on Training State:
AlphaRoute-VL-0.8B was created by surgically transplanting the pre-trained native ViT visual encoder and cross-attention weights (model.visual.*) from Qwen/Qwen3.5-0.8B-Base back into the hardened instruction-tuned text routing weights of AlphaRoute-0.8B-v1.5.

It has NOT yet undergone explicit end-to-end multimodal SFT fine-tuning on image-text pairs.

Despite having zero explicit multimodal routing supervision, this zero-shot stitched architecture demonstrates emergent visual perception: it successfully reads image pixels (OCR, UI modals, defect photos, scanned invoices), maps visual cues to arbitrary runtime categories, and extracts pixel-grounded entities into structured JSON fields.

---

🌟 Key Capabilities & Highlights

1. Dual-Mode Inference (Multimodal + Pure Text):
   • Multimodal Mode: Pass an image alongside instructions and schemas to perform visual intent classification and OCR slot extraction.
   • Pure Text Mode: When passed text-only queries (images=None), the vision encoder is completely bypassed. It retains 100% of AlphaRoute-0.8B-v1.5's state-of-the-art text routing performance with zero degradation.
2. Dynamic Runtime Ontologies: No fixed classes. Define candidate labels, descriptions, and JSON schemas on the fly per request.
3. Structured JSON Slot Filling: Directly extracts visual tokens (error codes, transaction IDs, monetary amounts, tracking numbers) from pixel buffers into designated schema keys.
4. Lightweight & Fast: At 0.8B parameters (~1.66 GB fp16/bf16 memory footprint), it runs at edge speeds on consumer hardware (MacBook M-series MPS, mobile devices, single low-cost cloud GPUs).

---

📊 Benchmark Evaluation 1: Text-Based Routing (100% Retained from v1.5)

Because pure text queries bypass the vision encoder, AlphaRoute-VL-0.8B retains the exact state-of-the-art benchmark scores of AlphaRoute-0.8B-v1.5:

---

📊 Benchmark Evaluation 2: Multimodal Visual Routing (300 Test Cases)

Evaluated on the Golden VL Benchmark (300 distinct procedural scenarios) across 6 domains with pixel-grounded truth:

Domain Breakdown (300 Cases)

• DevOps & Cloud SRE (50/50 - 100.0%): K8s OOMKilled (exit_code: 137), TLS cert expired (code: 526), DB pool exhaustion, Disk full (errno: 28), CrashLoopBackOff (SIGSEGV 139).
• FinTech & Payments (50/50 - 100.0%): AML cash structuring flags, SWIFT clearing receipts, Stripe card declines, B2B vendor invoices, chargeback disputes.
• Web & Mobile UI Bugs (50/50 - 100.0%): 504 Gateway Timeout modals, 403 Forbidden screens, 502 Bad Gateway, React runtime TypeError, 503 Maintenance windows.
• Logistics & Claims (50/50 - 100.0%): Crushed delivery parcel damage proof, address geofence mismatch, RMA return barcode labels, stalled hub scans, clean delivery proofs.
• Identity & KYC (50/50 - 100.0%): State driver licenses (valid vs expired), international passports with MRZ lines, corporate employee badges, unreadable flash glare audit rejections.
• Adversarial Out-of-Scope (36/50 - 72.0%): Procedural non-technical images (sunsets, geometric art, food illustrations) paired with technical schemas. Zero-shot flagged 72% as out_of_scope: true.

---

🚀 Quickstart & Usage

Installation

pip install torch transformers pillow accelerate

1. Multimodal Visual Routing (Image + Schema)

import torch
from PIL import Image
from transformers import AutoProcessor, AutoModelForImageTextToText

model_id = "NamanAgnih0tri/AlphaRoute-VL-0.8B"
device = "mps" if torch.backends.mps.is_available() else ("cuda" if torch.cuda.is_available() else "cpu")

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map=device
)

# Load input image
image = Image.open("path/to/server_error.png").convert("RGB")

task = "Triage server crash screenshot and extract error telemetry."
categories = {
    "k8s_oom_kill": "Out of memory container crash, exit code 137.",
    "tls_cert_expired": "Certificate expiration or handshake failure.",
    "db_pool_exhausted": "Database connection pool saturated."
}
schema = {
    "incident_type": "k8s_oom_kill | tls_cert_expired | db_pool_exhausted | null",
    "exit_code": "string | null",
    "failed_pod": "string | null",
    "out_of_scope": "boolean"
}

cat_lines = "\n".join([f"- {k}: {v}" for k, v in categories.items()])

prompt = f"""You are a precise semantic routing engine. Output only valid JSON.
TASK:
{task}
CATEGORIES:
{cat_lines}
OUTPUT SCHEMA:
{schema}
INPUT:
<|vision_start|><|image_pad|><|vision_end|>
"Please diagnose the attached error screenshot."
JSON:"""

inputs = processor(text=prompt, images=image, return_tensors="pt").to(device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=150, do_sample=False)

response = processor.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
# Expected Output:
# {"incident_type": "k8s_oom_kill", "exit_code": "137", "failed_pod": "payment-svc-129", "out_of_scope": false}

2. Pure Text Routing (Text Only)

prompt = """You are a precise semantic routing engine. Output only valid JSON.
TASK:
Route incoming fintech customer inquiry.
CATEGORIES:
- dispute_charge: Unauthorized transaction or card fraud claim.
- card_replacement: Damaged or lost card replacement request.
- wire_transfer: Inquiries about international SWIFT wire status.
OUTPUT SCHEMA:
{"route": "dispute_charge | card_replacement | wire_transfer | null", "out_of_scope": "boolean"}
INPUT:
"Someone used my debit card at an electronics store in Miami yesterday and I live in Chicago!"
JSON:"""

# When images is omitted, the vision tower is completely bypassed
inputs = processor(text=prompt, return_tensors="pt").to(device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=60, do_sample=False)

response = processor.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
# Output: {"route": "dispute_charge", "out_of_scope": false}

---

🛠️ Model Architecture Details

• Base Architecture: Qwen3_5ForConditionalGeneration (Qwen3-VL framework)
• Parameter Count: 0.8 Billion (~800M parameters)
• Vision Encoder: Native Qwen3.5 ViT with spatial merger (model.visual.*)
• Language Backbone: Qwen3.5 24-layer hybrid linear/full attention with rotary position embeddings
• Context Window: 262,144 tokens
• Weights Precision: bfloat16 safetensors (1.75 GB download)

---

📜 Citation & Attribution

@misc{alpharoute_vl_2026,
  author = {Naman Agnihotri},
  title = {AlphaRoute-VL-0.8B: Instruction-Conditioned Multimodal Semantic Routing SLM},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/NamanAgnih0tri/AlphaRoute-VL-0.8B}}
}