qwen3-1.7b-cs-support-gguf

Qwen3-1.7B Customer Support — Q4_K_M GGUF

The fine-tuned customer-support model, merged (Qwen3-1.7B + LoRA) and quantized to Q4_K_M for zero-cost CPU/Metal self-hosting (~1 GB). This is the exact artifact used for the reported evaluation and served behind the demo HTTP API.

• LoRA adapter (training-side): kvn12/qwen3-1.7b-cs-support-lora
• Base model: Qwen/Qwen3-1.7B

Results (held-out 1,100-item test split, this Q4_K_M build via Ollama, Opus-5 judge)

Better on every metric across all 11 categories / 27 intents.

Run with Ollama (recommended — exact no-think template)

A Modelfile reproducing the training template (empty <think></think>) is included:

huggingface-cli download kvn12/qwen3-1.7b-cs-support-gguf \
    tuned-qwen3-1.7b-Q4_K_M.gguf Modelfile --local-dir cs-tuned
cd cs-tuned
ollama create cs-tuned -f Modelfile
ollama run  cs-tuned "I was charged twice for order 8842, please help"

Chat via Ollama's HTTP API (send the system prompt + think:false):

curl localhost:11434/api/chat -d '{
  "model": "cs-tuned", "think": false,
  "options": {"temperature": 0, "seed": 42, "num_predict": 512},
  "messages": [
    {"role": "system", "content": "You are a customer support assistant for an online business. A customer has sent you a message. Respond directly to the customer in a helpful, professional, and empathetic tone. Understand what they need, give accurate information, and clearly explain any steps required to resolve their request. Keep the response focused and concise."},
    {"role": "user", "content": "how do I get a refund for a damaged item?"}
  ]}'

Run with llama.cpp

llama-cli -m tuned-qwen3-1.7b-Q4_K_M.gguf -p "<chat-formatted prompt>"

Prompt contract (train == serve)

Qwen3 chat format, thinking disabled; fixed system prompt (above); greedy decoding (temperature=0, seed=42, num_predict=512). Base and tuned use the same Modelfile — only the weights differ — so comparisons are apples-to-apples.

Provenance

Merged from the LoRA adapter into Qwen3-1.7B (fp16) and converted with llama.cpp convert_hf_to_gguf.py → llama-quantize Q4_K_M. The base model was quantized through the identical pipeline for the evaluation baseline.

Limitations

Reproduces the dataset's {{placeholder}} style; residual ~12% hallucination (worst on cancellation-fee queries); LLM-as-judge metrics (not human).