Triton-VX-Qwen3.5-2B-DPO

Triton VX Qwen3.5 2B (DPO)

A domain-adapted and DPO-aligned 2B parameter assistant for Fabrix.ai / RDAF knowledge retrieval, documentation citation, custom widget dashboards, storyboards, and dynamic pipelines.

This model was trained using two-stage alignment:

1. Supervised Fine-Tuning (SFT) on Fabrix technical documentation, schemas, and citation formats.
2. Direct Preference Optimization (DPO) using curated preference pairs to eliminate multi-line topic directory citations and enforce concise single-card citations (kb/cards/*.md).

---

📊 Evaluation & Alignment Results

Evaluated across the Fabrix citation preference benchmark:

Sample Output Comparison

User Query:

"Put a group filter on the top so I can pick site type, tier, and primary transport."

• ❌ SFT Baseline:

  kb/dashboards/attributes.md
  kb/widgets/tables.md
  kb/pstreams/flows.md
  

• ✅ This DPO Model:

  kb/cards/widget-filters.md
  

---

🚀 Usage with transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "Fabrix-AI-Inc/Triton-VX-Qwen3.5-2B-DPO"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
)

system_prompt = (
    "You are Fabrix cite-mode retrieval. Reply with repo-relative kb/ paths only, one per line. "
    "Prefer kb/cards/*.md. Cite a topic file (kb/pstreams/, kb/widgets/, kb/dashboards/, kb/ux/, kb/entry/) "
    "only when no card fits. Typically 1-5 paths. No prose, no HTML, no line numbers, no GET/POST examples, "
    "no secrets. Zero paths only if nothing under kb/ applies."
)

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "Put a group filter on the top so I can pick site type, tier, and primary transport."},
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=64,
        do_sample=False,
    )

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response.strip())
# Output: kb/cards/widget-filters.md

---

🏗️ Model Details

• Base Model: unsloth/Qwen3.5-2B
• Architecture: Qwen3_5ForConditionalGeneration
• Precision: 16-bit Float (torch.float16)
• Format: Standalone Merged Safetensors (model.safetensors)
• Framework: Unsloth / Hugging Face Transformers / TRL