qwen3-4b-guardrails-embedding-v1

Qwen3-4B Guardrails Embedding v1

Fine-tuned from Qwen/Qwen3-Embedding-4B for regulatory document retrieval as part of a Multi-Agent Debate (MAD) Guardrails system developed at SJSU.

This model serves as the retrieval backbone in a RAG pipeline that provides ground truth for agent debates on AI governance and regulatory compliance.

---

Model Description

---

Training Details

---

Evaluation Results

Evaluated on a 4-candidate reranking task (1 positive + 3 hard negatives) with 864 val / 864 test samples.

Overall

Per Query Type — Validation

Per Query Type — Test

---

Usage

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "vineeth453/qwen3-4b-guardrails-embedding-v1",
    trust_remote_code=True
)
model.max_seq_length = 512

# Queries require the instruction prefix
QUERY_INSTRUCTION = (
    "Instruct: Retrieve relevant regulatory passage to answer the query\n"
    "Query: "
)

# Passages are encoded without any prefix
query   = QUERY_INSTRUCTION + "What are the documentation requirements for high-risk AI systems?"
chunks  = [
    "EU AI Act Article 11 requires providers of high-risk AI systems to draw up technical documentation...",
    "NIST AI RMF suggests organizations establish governance structures for AI risk management..."
]

query_emb  = model.encode([query],  normalize_embeddings=True)
chunk_emb  = model.encode(chunks,   normalize_embeddings=True)
scores     = query_emb @ chunk_emb.T
print(scores)

Important: Always apply the instruction prefix to queries. Chunks/passages are encoded without any prefix. Consistency between fine-tuning and inference is critical for performance.

---

Intended Use

• Retrieval in RAG pipelines for AI governance and regulatory compliance
• Ground truth retrieval for multi-agent debate (MAD) systems
• Semantic search over regulatory documents (EU AI Act, NIST frameworks, cybersecurity regulations)

Out-of-Scope Use

• General-purpose semantic similarity (not optimized for non-regulatory domains)
• Generation tasks
• Classification without a retrieval head

---

Roadmap

• Round 2 fine-tuning — r=32, all 3 hard negatives, MNR scale=15, LR=5e-5, 4 epochs
• Hard negative re-mining using this model as the miner
• Quantization (GGUF / AWQ) for faster inference
• Larger eval pool (10-20 candidates for production-realistic metrics)

---

Project Context

This model is part of the SJSU Guardrails Project — a system for evaluating AI agent outputs against regulatory ground truth using Multi-Agent Debate (MAD) orchestration.

Pipeline: User query → RAG retrieval (this model) → Ground truth chunks → Agent debate → Guardrail verdict

---

License

Apache 2.0 — same as base model.