vectrayx-vision-1b

VectraYX-Vision-1B

A sub-2B vision–language model specialized for cybersecurity imagery — disassembly/decompiler panes (IDA, Ghidra), packet captures (Wireshark), scanner and post-exploitation terminals (Nmap, Metasploit), and memory-forensics dumps (Volatility) — that answers in Spanish, emits structured visual reasoning via native <|think|> tokens before its answer, invokes external tools through the Model Context Protocol using native <|tool_call|> tokens, and exports cleanly to llama.cpp's LLaVA mmproj format for fully on-premise, air-gapped deployment.

This is the GGUF export (model + mmproj) for offline inference. For the text-only backbone see jsantillana/vectrayx-1b.

Architecture

Files

• model.gguf (2.22 GB, F16) — the LLM decoder (also usable standalone for text as in vectrayx-1b)
• mmproj.gguf (823 MB, F16) — the SigLIP encoder + MLP projector (the vision tower)

Both are needed for multimodal inference; load both in llama.cpp / Ollama.

Training

Four-phase curriculum (all phases complete):

1. Phase 1 — 9.2B tokens general Spanish pretraining (backbone)
2. Phase 2 — ≈50B tokens three-block curriculum (backbone)
3. Phase 3 — ≈6B tokens tooling SFT establishing native <|tool_call|> + <|think|> (backbone)
4. Vision phase — 4a (projector alignment, ~228 steps), 4b (instruct, projector+backbone), 4c (think+tools). This checkpoint is the SFT-v2 continuation at step 400, warm-started from the v3B run (step 1900 of a low-LR phase-4b).

Total vision-phase compute: ≈2.2h of 2×A100-40GB wall time; ≈16M tokens.

Honest status

B6/B7 (vision benchmark) scores are near zero at the current training volume (400–1900 vision steps). The model generates fluent Spanish but ignores image content — visual grounding at this scale requires more SFT steps or a denser annotated screenshot corpus. This is a template/infrastructure release, not a quality release. Baseline context:

The text-only control confirms the benchmark is not trivially answerable from text alone — the with-image v3B row (0.08) exceeds the no-image control (0.02).

Usage (llama.cpp multimodal)

# Load model + vision tower
./llama-cli -m model.gguf --mmproj mmproj.gguf \
  --image /path/to/screenshot.png \
  -p "¿Qué herramienta de seguridad muestra esta captura y qué hace?" \
  --temp 0.7 -n 512

Usage (Ollama)

# Register the model with its mmproj
ollama create vectrayx-vision-1b \
  -f <(printf 'FROM ./model.gguf
PARAMETER temperature 0.7
PARAMETER num_predict 512
')
# (mmproj must be co-located for multimodal; see Ollama LLaVA docs)
ollama run vectrayx-vision-1b

Notes / known limitations

• Visual grounding is weak. The model is primarily an architecture/data/curriculum release pending a longer SFT run. Do not expect useful visual QA yet.
• NoPE export: NoPE layers export as RoPE layers with rotary application disabled per layer. The GGUF RoPE permute fix uses stack([w1,w2], dim=2) interleaved.
• Export is GGUF-splittable: the decoder checkpoint contains no visual weights and vice versa, matching the LLaVA model.gguf / mmproj.gguf convention.
• This is the first released model to combine periodic NoPE layers with an injected visual-token block; the NoPE×vision interaction is an open research question (see the accompanying paper for the ablation design).

Citation

See the accompanying paper, VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use. Paper on Hugging Face.

Author

Juan S. Santillana — DevOps engineer (Globant). Institutional affiliation approval pending. Weights, benchmarks, and configs are released; the tool-calling MCP Modelfile will follow the visual-prompt-injection and over-triggering safety evaluation.