uyscout-gemma3-270m-lora

uyscout-gemma3-270m-lora

A LoRA fine-tune of google/gemma-3-270m-it, merged into the base weights and exported to ONNX (dynamic INT8 quantization), built for UyScout — a Tashkent property-research chatbot that runs entirely client-side in the browser via WebGPU/onnxruntime-web.

What this fixes

The stock gemma-3-270m-it reliably avoids inventing entirely new facts, but tends to answer generically instead of citing the specific POI names / price figures given in its context — e.g. "a major shopping and transit hub" instead of naming "Chorsu Bazaar" when that name is right there in the prompt.

This checkpoint was fine-tuned (LoRA, r=8, q_proj/k_proj/v_proj/o_proj, 30 epochs) on 14 hand-written examples that always cite exact proper nouns and numbers from the given fact sheet.

Measured result (not asserted)

On a genuine held-out set of 14 new examples never seen during training (different districts, different POI categories — gym/university/mosque/bank/pharmacy, never appeared in training):

Paired with a deterministic post-generation fact-injection safety net (splices in any real POI name / price figure the model still leaves out — can only ever add grounded content, never fabricate). Full methodology, the earlier techniques that were tried and ruled out (Context-Aware Decoding, DAGCD/attention-guided decoding), and every real bug found along the way are documented at github.com/SRKRZ23/uyscout (bench/ harness).

Files

• model_int8.onnx — merged + INT8-quantized ONNX graph (dynamic quantization via onnxruntime.quantization.quantize_dynamic), single self-contained file, ~418MB.
• Standard tokenizer/config files, unchanged from the base checkpoint.

Usage

Same interface as any ONNX-exported Gemma3ForCausalLM — input_ids, attention_mask, and past_key_values.{i}.{key,value} (float32, num_key_value_heads=1, head_dim=256, num_hidden_layers=18). See on-device-llm.js in the UyScout repo for a working onnxruntime-web/WebGPU integration.

License

Inherits the Gemma Terms of Use from the base model — this is a fine-tuned derivative, not a from-scratch model.