azka-2026-Q3-v1

Azka 2026-Q3-v1 — INTERNAL / UNREVIEWED

Not clinically reviewed. Not for public, patient, or production use. Release is gated on a qualified health-care reviewer signing off on safety-scenario responses (data/eval/safety_prompts.jsonl), and that review has not happened yet. This repo exists only for internal testing by the team.

What this is

An on-device Q&A assistant for pregnancy and infant-care questions (Indonesian), fine-tuned to answer from reference guidance and to redirect warning-sign questions to a clinician rather than attempt a diagnosis.

Training

Data pipeline: raw conversation history → automated filtering (dedup, template/boilerplate removal) → PII scrub + human-reviewed privacy pass → train split. No synthetic/guideline-authored QA is merged into this cycle's training set yet (available but held out — provenance not yet verified against source documents).

Files

• model-2026-Q3-v1-Q4_K_M.gguf — merged + quantized, ~372MB, GGUF format for on-device llama.cpp-based inference (this is what the app ships)

Quick test (desktop)

llama-cli -m model-2026-Q3-v1-Q4_K_M.gguf -p "Mual di pagi hari saat hamil muda itu normal tidak?" -n 220 --temp 0.5

Uses the model's built-in chat template automatically. Conservative generation settings used in-app: temp=0.5, n=220 tokens.

Implementing in the Android app

The app (gemma_medical_app, Flutter) already runs local GGUF inference via the fllama plugin (llama.cpp FFI wrapper, Android + iOS support). To swap in this build:

1. Drop model-2026-Q3-v1-Q4_K_M.gguf where the app expects its model file (see lib/features/model/ — model download/path resolution).
2. fllama loads the GGUF directly — no conversion needed, same format already used by llama-cli above.
3. Keep the app's existing conservative generation params (temp=0.5, capped output tokens) — that's what this build was validated against during eval, not the defaults.
4. Ship only after the clinician safety review passes — see "Known limitations" below. Swapping the file in is a 5-minute change; the review gate is not.

Accuracy & validation status

No accuracy number exists for this build. This is the result of the initial training run only — capability has not been measured yet (capability.measured = false in the pipeline's own status record).

Measurement method (defined, not yet run to completion):

• QA eval — held-out question set (data/eval/general_qa.jsonl, draft 39/175 target items), scored by an LLM-judge pre-filter, never authoritative on its own
• Safety eval — red-flag scenario set (data/eval/safety_prompts.jsonl, draft 20/40 target items), zero-tolerance: one wrong answer here blocks release regardless of QA score
• Clinician review (step 2, not started) — a qualified health-care reviewer scores real model responses against the safety set; this is the authority, not the LLM-judge pre-filter above. This step hasn't run yet — it's the next step before any accuracy claim or release decision.

Known limitations

• Safety eval set is a draft (20/40 target scenarios written, none scored yet)
• QA eval set is a draft (39/175 target)
• No expert/clinician review on file for this cycle — this is the reason it isn't released, not a formality
• Guideline-authored QA (76 items) exists but isn't merged into training — authorship/provenance unverified