ngari-ft-distilled

NGARi FT Distilled — Sovereign QA 1.5B

A 1.5B distilled QA model fine-tuned on NGARi's sovereign-agent domain data — fast enough to run as a real-time content-safety judge and QA model on aarch64 edge hardware with 8GB RAM, with zero cloud dependency.

This model is the production guardrail judge inside the NGARi Sovereign Business Operating System (NS-BOS): every agent response passes through it before delivery.

Why it exists — the NGARi model pipeline

NGARi's architecture pairs a large teacher model with small, deployable edge models:

Teacher (27B-class, e.g. qwen3.8-27B)
    │  generates reasoning traces, synthetic data, eval judgments
    ▼
Edge models (1.5B–2B: ngari-ft-distilled, ngari-tool)
    │  distilled / fine-tuned on teacher outputs
    ▼
Deployment: air-gapped edge hardware (Jetson AGX Orin, 8GB RAM)

This repo is the distilled student in that pipeline — capabilities that normally need a much larger model, compressed into a 1.5B footprint that runs entirely on owned hardware.

Provenance (verified Aug 3, 2026)

Google Gemma models were served only on NGARi hardware and were never used in NGARi training. All training used the Apache-2.0 Qwen2.5 lineage.

Evaluation

Chat quality — ngari-ft-distilled_chat_eval.json

{
  "model": "ngari-ft-distilled",
  "num_examples": 200,
  "avg_score": 0.3766,
  "avg_latency_sec": 2.81,
  "tokens_per_sec": 39.78,
  "total_time_sec": 561.98
}

Tool detection (secondary capability) — ngari-ft-distilled-stable_tool_eval.json

{
  "model": "ngari-ft-distilled:stable",
  "num_examples": 20,
  "tool_detection_rate": 0.6,
  "tool_name_accuracy": 0.55,
  "params_validity_rate": 0.6,
  "avg_latency_sec": 2.38
}

For high-accuracy tool calling, use NGARiAI/ngari-tool (100% on all three tool metrics). This model's role is QA + safety judging.

Files

Usage

# Ollama (GGUF)
ollama create ngari-ft-distilled -f Modelfile
ollama run ngari-ft-distilled "your prompt"

# Transformers (merged)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("NGARiAI/ngari-ft-distilled")

# PEFT adapter (apply on the base)
from peft import PeftModel
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-1.5B-Instruct")
adapter = PeftModel.from_pretrained(base, "NGARiAI/ngari-ft-distilled")

Companion repos

• NGARiAI/ngari-tool — tool-calling sibling (100% tool-format eval)
• NGARiAI/ngari-datasets — the training data
• NGARiAI/ns-bos-kernel — the Apache 2.0 sovereign kernel

Sovereign AI

Trained and verified on user-owned edge hardware with zero cloud dependency. Verified air-gap (monitored via /proc/net/dev). "AI You Own. Completely."