Selora-AI-LLM-14B

Selora Homes: selorahomes.com Selora AI Home Assistant Integration: github.com/SeloraHomes/ha-selora-ai

---

Selora AI LLM 14B

Selora AI LLM is an instruction-tuned language model for Home Assistant: five small LoRA specialists on one shared base, hot-swapped per request by the Selora AI integration. This is the larger line, on Qwen3 14B, for deployments with about 16 GB of memory, and the model for Selora Hubs with 32 GB of RAM. The 1.7B line runs on far less.

• command — control devices ("turn off the kitchen lights")
• automation — author home automations and blueprints
• answer — answer questions about the home's live state
• clarification — ask a follow-up when a request is ambiguous
• utilities — docs-grounded maintenance, troubleshooting, and setup help

Both lines are trained on the same corpus, against the same system prompts, and emit the same compact JSON slim envelopes, so everything the 1.7B card documents about the specialists, their output shapes, and setting the model up in Home Assistant applies here unchanged. This card covers what differs: the files, the memory, and the measured scores.

A single fused model for runtimes that cannot swap adapters is published separately as selorahomes/Selora-AI-LLM-14B-ollama.

Base: Qwen3 14B · Format: GGUF Q5_K_S base + 5 per-specialist LoRA adapters (F16) · License: Apache-2.0

Results

Measured on the open allenporter/home-assistant-datasets suite — the harness behind the Home Assistant LLM leaderboard — at temperature 0, over the full suite, with llama.cpp on CUDA at the shipping quant.

Two modes are reported, because they measure different things:

• model only — the integration's deterministic overrides are switched off, so every turn is decided by the model's own output. This is the number a retrain moves.
• net + model — those overrides in the loop. Close to the shipped path, not identical: approval and the safe-command allowlist are off for the run, and questions reads high on calendar cases, whose fixtures supply event data a real Home Assistant does not.

95% Wilson intervals. automations is four blueprint tasks, five samples each, scored separately because the harness collects those replies without checking them: a reply passes if Home Assistant accepts it as an automation blueprint with every requested input declared and wired. The automation is not run.

The fused single-model build scores lower, most of all with the overrides off; its card has the numbers.

Files

The five specialists share one base, so the whole set costs about 10.5 GB of weights rather than five separate 14B models.

Requirements

Why Q5_K_S

The quant is chosen to fit a 16 GB deployment with the adapters, the KV cache and Home Assistant running alongside. Q4_K_M is smaller, but measurably hurt documentation-lookup accuracy.

Running it

llama-server (Home Assistant integration runtime)

Download the base and all five adapters into one directory, then:

llama-server \
  --model qwen3_14b_base.Q5_K_S.gguf \
  --lora-init-without-apply \
  --lora selora-command.f16.gguf,selora-automation.f16.gguf,selora-answer.f16.gguf,selora-clarification.f16.gguf,selora-utilities.f16.gguf \
  --ctx-size 4096

One --lora with comma-separated paths, not one flag per adapter: llama.cpp keeps only the last value when --lora is repeated, so the per-flag form loads a single adapter and silently answers every intent with it.

POST to /lora-adapters to switch the active adapter before each /v1/chat/completions call. Resolve slot ids by filename from the /lora-adapters listing rather than assuming an order.

Ollama

From the same directory, one named model per specialist:

ollama create selora-14b-command       -f Modelfile.command
ollama create selora-14b-automation    -f Modelfile.automation
ollama create selora-14b-answer        -f Modelfile.answer
ollama create selora-14b-clarification -f Modelfile.clarification
ollama create selora-14b-utilities     -f Modelfile.utilities

Ollama does not hot-swap LoRAs in-process, so this path suits direct chat and scripting; for the Home Assistant integration use llama-server. For a single model instead of five, use the fused build.

Prompt format

Training and inference must match byte-for-byte. ChatML, with a /no_think prefix on the user turn so Qwen3 returns bare JSON:

<|im_start|>system
{system prompt}<|im_end|>
<|im_start|>user
/no_think {user message}<|im_end|>
<|im_start|>assistant
<think>

</think>

The empty <think></think> block is part of the prompt, not the output, and training includes it (masked). The chat template embedded in these GGUFs always emits it, so llama.cpp and Ollama supply it whatever the client asks for. The user message carries USER REQUEST, EXISTING AUTOMATIONS, IMPORTANT, and AVAILABLE ENTITIES, in that order; utilities adds RELEVANT DOCS. The 1.7B card has the details.

Generation parameters

{
  "temperature": 0.0,
  "repeat_penalty": 1.0,
  "repeat_last_n": 256,
  "max_tokens": 384,
  "stop": ["<|im_end|>", "<|endoftext|>"]
}

Bump max_tokens to 1536 for automation requests.

Training

Base: Qwen3 14B — the post-trained checkpoint, not Qwen3-14B-Base. Fine-tuned with PEFT on CUDA as QLoRA (4-bit base, paged_adamw_8bit, learning rate 5e-5, effective batch 8, sequence length 4096). Each specialist is its own LoRA, trained on a synthetic Home Assistant corpus generated from curated home specifications and a service matrix — no scraped forum or user data — with loss on the assistant turn only.

manifest.json records the exact recipe of the release you are on: steps, epochs and example count per specialist, held-out eval loss, and the software stack each adapter was trained with.

Versioning

main is the latest release. Each release is also tagged v<x.y.z>, and manifest.json records the version, per-file checksums and the training recipe for whatever revision you are on.

License

Apache 2.0, inherited from Qwen3-14B. See LICENSE and NOTICE.