gemma-3-1b-kbd-q4km

Gemma 3 1B — Context-Aware Keyboard (Q4_K_M GGUF)

A fine-tuned Gemma 3 1B (INT4) for context-aware text prediction on Android, quantized to GGUF Q4_K_M for on-device inference via llama.cpp.

Model Details

• Base model: google/gemma-3-1b-pt
• Fine-tuning: LoRA via Unsloth on a single RTX 3060
• Quantization: Q4_K_M via llama.cpp convert_hf_to_gguf.py
• Target device: Android (HeliBoard keyboard fork)

Training Data

~3B token conversational corpus from five sources with per-source cleaning, thread reconstruction, and synthetic typo augmentation:

• Reddit conversations
• OpenSubtitles
• Enron email corpus
• Hacker News comments
• MS MARCO queries

Prompt Format

The model uses custom special tokens to condition predictions on app context:

<|app|>messaging<|ctx|>
[on-screen conversation history]
<|cursor|>

• <|app|> — app category (messaging, email, search, social, etc.)
• <|ctx|> — marks start of on-screen conversational context
• <|cursor|> — marks the cursor position; model predicts the next tokens

Evaluation

Evaluated on 1,865 examples.

Usage

Designed for integration with HeliBoard via llama.cpp's Android NDK build. Load the GGUF, feed the structured prompt, and use top-k token predictions as keyboard suggestions.

Limitations

• English-only
• Trained on informal/conversational text; may underperform on formal or technical writing
• ~800MB model size requires download on first app launch