vectrayx-nano

VectraYX-Nano

VectraYX-Nano is a 42M-parameter Spanish cybersecurity language model trained from scratch with curriculum learning and native Model Context Protocol (MCP) tool use. It is, to our knowledge, the first published Spanish-native cybersecurity LLM with end-to-end MCP integration.

• Paper: VectraYX-Nano: A 42M-Parameter Spanish Cybersecurity Language Model with Curriculum Learning and Native Tool Use
• Repository: vectrayx/vectrayx-nano-paper
• arXiv DOI: https://doi.org/10.48550/arXiv.2605.13989
• Author website: https://jsantillana.com

---

Released Model: VectraYX-Nano v7 (Headline)

VectraYX-Nano v7 is the released headline model. It uses the same 42M architecture and three-phase curriculum pre-training as the v2 bootstrap-ablation reference, with the SFT corpus rebalanced to a tool-use ratio of 1:21 (vs. 1:211 in v2). This single change raises B4 (tool-selection) from 0.000 to 0.230 ± 0.052 across N=4 seeds while retaining strong CVE recall (B1=0.332±0.005) and conversational quality (B5=0.725±0.130).

Files in this repo:

---

Key Results (VectraYX-Bench, N=4 seeds)

†B2 is a benchmark artifact in this revision (key mismatch in harness, fix queued).

B5 inversion: Nano v7 (0.725±0.130) and Nano v2 (0.775±0.043) both exceed GPT-4o (0.631) on the 314-prompt held-out chat suite — the register-matched bootstrap corpus makes conversational Spanish the model's first language.

---

Key Findings

1. Loss-vs-register inversion. A higher-perplexity bootstrap corpus (OpenSubtitles-ES) yields better post-SFT chat behavior than a lower-perplexity alternative (mC4-ES). At the nano scale, the bootstrap corpus dictates the model's default response style; SFT cannot fully overwrite it.

2. Tool-use is corpus-density-gated, not capacity-gated. The B4=0.000 floor in the mixed SFT (ratio 1:211) is a corpus-density artifact. Rebalancing to 1:21 (2,801 tool-use examples) shifts the first-token prior to <|tool_call|> and raises B4 to 0.230±0.052 at 42M — without retraining the backbone.

---

Inference: llama.cpp / Ollama (GGUF)

# With Ollama
ollama run hf.co/jsantillana/vectrayx-nano:vectrayx-nano-f16.gguf

# With llama.cpp
./llama-cli -m vectrayx-nano-f16.gguf \
  --chat-template llama3 \
  -p "<|system|>Eres VectraYX, asistente experto en ciberseguridad para LATAM.<|end|>" \
  -i

Runs at 6–10 tok/s on Raspberry Pi 4 and 60–100 tok/s on a laptop CPU.

---

Inference: PyTorch

from huggingface_hub import hf_hub_download
import torch, json, sys

sys.path.insert(0, ".")  # needs training/transformer.py from vectrayx-paper-code

ckpt = hf_hub_download("jsantillana/vectrayx-nano", "nano_sft_v7_s42.pt")
tok  = hf_hub_download("jsantillana/vectrayx-nano", "tokenizer/vectrayx_bpe.model")
cfg  = hf_hub_download("jsantillana/vectrayx-nano", "configs/nano.json")

Full inference script at vectrayx-paper-code.

---

Training Details

---

Citation

@misc{santillana2026vectrayx,
  title     = {VectraYX-Nano: A 42M-Parameter Spanish Cybersecurity Language Model
               with Curriculum Learning and Native Tool Use},
  author    = {Santillana, Juan S.},
  year      = {2026},
  eprint    = {2605.13989},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url       = {https://arxiv.org/abs/2605.13989}
}