qwen3_5_2B_voice_assistant-GGUF

Qwen3.5-2B Voice Assistant

Fine-tuned Qwen3.5-2B for voice assistant / conversational use.

This is designed to be short responses without thinking.

Trained on curated, concise datasets — all assistant responses are short and natural-sounding, optimized for spoken output rather than written text.

Training Details

Datasets

All datasets are filtered for concise, voice-friendly assistant responses (20–400 chars for general data, 20–500 chars for reasoning). Responses containing markdown formatting (bold, inline code, numbered lists, bullet points, headings) are excluded. Exact-match deduplication is applied across all sources before training.

Filtering Pipeline (v7)

Each assistant response is checked against the following before inclusion:

• Length: 20–400 chars (general), 20–500 chars (reasoning)
• No markdown: **bold**, `inline code`, [link](url), # headings all excluded
• No lists: numbered (1.) and bullet (-, *) patterns excluded at line-start and after colons
• No list lead-ins: phrases like "the process involves:", "as follows:", "the following" excluded
• No AI-isms: "certainly!", "as an AI", "in conclusion", "delve" excluded
• Post-dedup sanity check: % of markdown patterns logged to W&B before training

System Prompt

All training samples include this system prompt:

You are a casual, hands-free voice assistant. Speak in short, punchy sentences as if we are having a real-time conversation. Never use bullet points, markdown, or code. If explaining a complex topic, use a simple, everyday analogy. Respond immediately without any preamble or internal monologue.

Available Formats

Usage with llama.cpp

huggingface-cli download cowWhySo/qwen3_5_2B_voice_assistant-GGUF --include "*q4_k_m*" --local-dir .
./llama-cli -m *q4_k_m*.gguf --ctx-size 2048 --temp 0.7 --top-p 0.9

Usage with Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("cowWhySo/qwen3_5_2B_voice_assistant")
tokenizer = AutoTokenizer.from_pretrained("cowWhySo/qwen3_5_2B_voice_assistant")

messages = [
    {"role": "system", "content": "You are a casual, hands-free voice assistant. Speak in short, punchy sentences as if we are having a real-time conversation. Never use bullet points, markdown, or code. If explaining a complex topic, use a simple, everyday analogy. Respond immediately without any preamble or internal monologue."},
    {"role": "user", "content": "What's the weather like today?"}
]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True)
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Fine-tuned with Unsloth on an NVIDIA L4 GPU.