gemma-4-E2B-home-assistant

Gemma 4 E2B — Home Assistant

A fine-tune of Google's gemma-4-E2B-it-qat-q4_0-unquantized for controlling a smart home through natural language, in the style of the acon96/home-llm project. Given a system prompt describing the available devices and services, the model replies in natural language and emits the tool call needed to carry out the request.

This is the LoRA adapter merged back into the base weights — a complete, drop-in replacement for the base checkpoint. Like the upstream model it is the full multimodal Gemma4ForConditionalGeneration (text + vision + audio towers); only the text/language-model projections were fine-tuned, so the vision and audio towers are byte-for-byte the base model's.

Base model

• google/gemma-4-E2B-it-qat-q4_0-unquantized
• The -qat-q4_0-unquantized checkpoint is quantization-aware-trained and calibrated for llama.cpp Q4_0. Fine-tuning was done in 16-bit LoRA (not QLoRA/NF4) specifically to keep that QAT calibration intact, so the merged model still quantizes cleanly to Q4_0 for llama.cpp / Ollama deployment.

Training

Only the assistant turn (its text and its tool call) is trained; injected tool results are masked out so the model learns to call tools, not to hallucinate their responses.

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "mewse/gemma-4-E2B-home-assistant"
tok = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto")

messages = [
    {"role": "system", "content": "<your Home Assistant system prompt: devices, services, tool schemas>"},
    {"role": "user", "content": "turn on the living room lamp"},
]
enc = tok.apply_chat_template(messages, add_generation_prompt=True,
                             return_tensors="pt", return_dict=True).to(model.device)
out = model.generate(**enc, max_new_tokens=128, do_sample=False)
print(tok.decode(out[0][enc["input_ids"].shape[1]:], skip_special_tokens=False))

For the best tool-call fidelity, match the system-prompt format used in training (entity states + tool schemas injected into the system prompt, as in home-llm). For deployment, quantize to a GGUF (Q4_0 is the QAT-native target) and serve via llama.cpp or Ollama.

License

Governed by the Gemma Terms of Use, inherited from the base model. You must accept the Gemma license to use these weights.