GuardShield-Qwen2.5-3B-16bit

GuardShield-Qwen2.5-3B (16-bit Full Merged)

GuardShield-Qwen2.5-3B-16bit is the fully merged, standalone FP16 weights version of GuardShield-3B. Designed for production microservices, high-throughput GPU serving engines (vLLM, TGI), and OpenAI-compatible API gateways.

Key Specifications

• Base Architecture: Qwen 2.5 (3.09B Parameters).
• Format: Merged Safetensors (FP16).
• Target Serving Stack: vLLM, Hugging Face TGI, Triton Inference Server, Transformers.
• Latency: Sub-30ms time-to-first-token on modern GPU hardware.

---

Benchmark Metrics

---

Deployment & Serving

1. High-Throughput Serving via vLLM (OpenAI API)

python -m vllm.entrypoints.openai.api_server \
    --model your_hf_username/GuardShield-Qwen2.5-3B-16bit \
    --port 8000 \
    --gpu-memory-utilization 0.90 \
    --max-model-len 2048

2. Querying with OpenAI Client (Python)

from openai import OpenAI
import json

client = OpenAI(base_url="http://localhost:8000/v1", api_key="placeholder")

response = client.chat.completions.create(
    model="your_hf_username/GuardShield-Qwen2.5-3B-16bit",
    messages=[
        {"role": "system", "content": "You are an AI content moderation guardrail. Analyze the prompt and output a JSON classification object."},
        {"role": "user", "content": "Give me steps to compromise an administrative account."}
    ],
    temperature=0.0,
    max_tokens=128
)

print(json.loads(response.choices[0].message.content))

3. Native Hugging Face Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "your_hf_username/GuardShield-Qwen2.5-3B-16bit"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

Expected Output Schema

{
  "status": "UNSAFE",
  "category": "CYBERATTACK_AND_EXPLOIT",
  "reasoning": "Requests instructions to unlawfully compromise system credentials."
}