lastbox-gemma4-e2b-sft-v3

LastBox — Gemma 4 E2B fine-tuned for offline survival assistance (v3 SFT, submission snapshot)

Tiny offline survival assistant in a Pelican case. Gemma 4 E2B fine-tuned with Unsloth, quantized to Q4_K_M, served by llama.cpp on an 8 GB Raspberry Pi 5 over a LoRa 868 MHz mesh radio. ~700 ms first token, 6–7 tok/s sustained on ARM CPU.

This is the v3 SFT snapshot submitted to the Kaggle "Gemma 4 Good" Hackathon 2026 (deadline 2026-05-18 23:59 UTC). For the post-deadline v6 SFT-warmup checkpoint with 72 % tool-emission and 38× agentic_score lift, see norecyc/lastbox-gemma4-e2b-v6-toolprior.

• GitHub: https://github.com/agentsmill/lastbox (tag v1-submission)
• Live site: https://agentsmill.github.io/lastbox/

What it does

1. Survival Q&A — text-only first aid, bushcraft, navigation, power, hazards.
2. Optical triage — paired with mmproj-F16.gguf (SigLIP vision, unmodified upstream) it describes plants, wounds, terrain from the RPi camera.
3. Mesh-radio relay — every reply destined for the radio is hard-capped at 150 bytes UTF-8 so it fits a single LoRa packet at the legal duty cycle.

Files

Quickstart — llama.cpp server

docker run --rm -p 11436:8080 \
  -v $(pwd):/models:ro \
  ghcr.io/ggml-org/llama.cpp:server \
  -m /models/lastbox-gemma4-e2b-v3-q4_k_m.gguf \
  --host 0.0.0.0 --port 8080 \
  --ctx-size 4096 --threads 4 --parallel 1 \
  --jinja --chat-template-file /models/chat_template.jinja \
  --no-mmap --mlock

Training recipe

base:               unsloth/gemma-4-E2B-it
framework:          Unsloth FastModel + TRL SFTTrainer
LoRA:               r=8, alpha=8, dropout=0, bias="none"
target modules:     language layers + attention + MLP, no vision
chat template:      gemma-4 (no-thinking) — suppresses CoT preamble
hardware:           1× NVIDIA GB10 (Grace Blackwell, 121 GB unified, CUDA 13)
data:               1 034 EN survival dialogs (train) + 114 (val)
data generation:    Kimi K2.5 teacher via OpenRouter, ~$1.10 total
epochs:             2 (v3 no-think); 3 (v2 thinking, deprecated)
lr:                 2e-4, cosine, warmup 10 steps
batch / grad accum: 4 / 4
training time:      27 min on GB10
final train loss:   0.08, eval loss plateau ~2.64 on 114-dialog held-out

Eval (golden_en, n=25 held-out, streaming SSE methodology)

Known limitations of this v3 snapshot:

• tool_emission_rate is ~0% — the SFT optimised on tool-call traces but the eval-time prompt was missing the tool-definitions JSON block from the training format. Fixed in v6 along with a targeted SFT warmup on tool-only pairs.
• Streaming-SSE eval lost 12/25 dialogs to client-side disconnects, so the 0.52 floor on byte/format/persona is actually 13/25 completion ratio, not a quality issue. Non-streaming eval shows every completed dialog passes.

Dataset

Training and eval data: norecyc/lastbox-survival-dialogues

License

Model weights are subject to the Gemma Terms of Use. Training scripts and LoRA adapter weights are Apache 2.0 (see GitHub repo).

Citation

@misc{lastbox_gemma4_v3_2026,
  title  = {LastBox v3: Gemma 4 E2B fine-tuned for offline survival assistance on Raspberry Pi 5},
  author = {Mateusz Pawelczuk},
  year   = {2026},
  url    = {https://huggingface.co/norecyc/lastbox-gemma4-e2b-sft-v3},
  note   = {Kaggle Gemma 4 Good Hackathon 2026 submission}
}