Little-Titles-GGUF

Little Titles GGUF

This repository contains GGUF quantizations of acon96/Little-Titles, a fine-tuned Qwen/Qwen3.5-0.8B-Base model for producing concise, descriptive titles for user requests. GGUF files are intended for llama.cpp and other GGUF-compatible runtimes.

Input contract

The embedded chat template accepts exactly one user message. It may be preceded by one system message. Multi-turn conversations, prior assistant messages, and tools are not supported. To title a complete conversation, serialize or summarize it into the content of that one user message.

When no system message is supplied, the template uses the instruction:

Generate a short title describing the following user request.

Local API deployment with llama.cpp

Start a local OpenAI-compatible server:

llama-server -hf acon96/Little-Titles-GGUF:Q8_0
  --host 0.0.0.0 \
  --port 8000

Then submit one user message to /v1/chat/completions:

curl http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Help me plan a three-day trip to Kyoto."}],
    "temperature": 0.8,
    "max_tokens": 32
  }'

Source model and training

These GGUF files are quantized derivatives of acon96/Little-Titles. The source model's training configuration is available at training-config.yml. It fine-tunes on SupraLabs/chat-titles-filtered-115K using Axolotl.

Evaluation

We compare Little Titles with a reference title supplied by the ogrnz/chat-titles dataset. Gemma 4 26B A4B rated each pair for accuracy, relevance, and concise usefulness.

Across 10,000 paired judgments, Little Titles won 4,831 comparisons versus 3,531 reference-title wins (1,638 ties). Its decisive-pair win rate was 57.8% (95% CI: 56.7%–58.8%; exact one-sided sign test $p = 2.81 \times 10^{-46}$), so it was preferred more often than the reference title in this evaluation.

This is exploratory evidence: Little Titles was always Candidate A, so candidate-position bias may affect this result.