phila-ft-v2-GGUF

phila-ft-v2

QLoRA fine-tune of Llama 3.2 3B Instruct for the phila speak gate - a silence-first iMessage group chat agent.

What this model does

Binary classification: given a batch of group chat messages, decide SPEAK or SILENT. The model is tuned to default to silence and only speak for factual corrections, unanswered questions, or direct address.

Training

• Base model: unsloth/Llama-3.2-3B-Instruct
• Method: QLoRA, r=16, alpha=16, dropout=0
• Training data: 1,138 examples across 4 targeted categories (755 base + 150 speak-unanswered + 153 silent-sarcasm + 80 near-miss)
• Hardware: Vast.ai RTX 4090, 429 steps, ~40 minutes
• Quantization: Q4_K_M GGUF

Results

The fine-tune specifically fixed the "buried thread" failure category (question asked, topic changes, nobody answers) which was a 0% pass rate across all prompt engineering attempts on 4 base models.

Usage with Ollama

# download the GGUF
# then create with the included Modelfile:
ollama create phila-ft-v2 -f Modelfile

# or manually:
ollama create phila-ft-v2 -f - <<EOF
FROM ./phila-ft-v2.Q4_K_M.gguf
TEMPLATE \"\"\"{{- if .System }}<|start_header_id|>system<|end_header_id|>

{{ .System }}<|eot_id|>{{ end }}{{- range .Messages }}<|start_header_id|>{{ .Role }}<|end_header_id|>

{{ .Content }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>

\"\"\"
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
PARAMETER temperature 0.1
PARAMETER top_p 0.52
PARAMETER num_predict 64
EOF

More info

• Full project: github.com/onblueroses/phila
• Research log: FINDINGS.md