AraUni-Granite-97M-Intent-Classifier

AraUni Granite 97M Intent Classifier

Release: v0.4.1 · Lifecycle: preview

This is a 20-label Arabic university intent classifier fine-tuned from ibm-granite/granite-embedding-97m-multilingual-r2. It routes one question to zero, one, or several university-information labels. The model uses first_token pooling and independent sigmoid outputs; it is not a generative model.

Synthetic-data warning: All training, validation, and test examples are synthetic. The reported scores do not guarantee that the model will perform well on real users' queries. Manually test the model with representative real queries before using it in any application.

Model details

• Architecture: ibm-granite/granite-embedding-97m-multilingual-r2 encoder + dropout + linear multi-label head
• Pooling: first_token
• Maximum input length: 256 tokens
• Model release: v0.4.1
• Release lifecycle: preview
• Weights: SafeTensors
• License: apache-2.0
• Training data composition: fully synthetic; no real-user queries
• Training data: NajahUniv/arabic-univeristy-chatbot-qa
• Dataset release: v0.4.0
• Pinned dataset revision: 0cbdab9e809ed8c2c3a06c467887a4f51188007f
• Decision thresholds: selected on validation data and stored in config.json and thresholds.json

The repository includes custom Transformers modeling code so it loads through the standard AutoClass API. Review the code in this repository before enabling trust_remote_code; in a production deployment, pin revision to a reviewed model commit SHA.

Release notes

v0.4.1 publishes the refreshed Granite-only model lineup trained on dataset release v0.4.0. The model card now compares the fast/edge 97M model with the highest-accuracy 311M model and makes the fully synthetic nature of the evaluation data explicit. It also includes validated CPU INT8 ONNX, ARM ORT mobile, and model-specific mixed-precision browser artifacts. The label taxonomy and inference contract are unchanged.

Evaluation

Threshold selection used only the validation split. The test split remained held out until the final comparison. Full aggregate and per-label results are available in validation_metrics.json and test_metrics.json.

Choosing between the Granite models

Granite 97M is the recommended fast/edge model when download size, memory use, latency, and local-device deployment matter most. Granite 311M is the highest-accuracy model in the published AraUni Granite lineup and is recommended when predictive quality is the priority. Both models use the same labels, data splits, threshold-selection procedure, and evaluation protocol, so the accuracy results above are directly comparable.

Local inference speed benchmark

MPS

CPU

Measured on Apple M5 Max (PyTorch 2.13.0, Transformers 5.14.1). Each batch size used 5 warm-up and 30 measured iterations over the same deterministically shuffled questions derived from the test split of NajahUniv/[email protected], pinned to commit 0cbdab9e809ed8c2c3a06c467887a4f51188007f, after applying the repository's standard dataset-preparation pipeline.

Numbers include tokenization, padding, device transfer, model forward pass, sigmoid, thresholding, and result construction. They exclude model loading and HTTP overhead. Latency and throughput depend on hardware, software versions, input lengths, batch size, and thermal state; compare only rows from this same run. Peak RSS is whole-process memory and MPS uses unified memory.

The full machine-readable benchmark report is included as benchmark_results.json. Use the direct benchmark for model-to-model speed comparisons; use the FastAPI load test below to size a specific deployment.

Optional CPU INT8 server inference

The onnx/ directory contains a dynamic-shape, dynamic-INT8 ONNX model for CPUExecutionProvider. It is a deployment alternative, not a replacement for the canonical SafeTensors checkpoint. onnx/onnx_config.json pins the graph checksum, label order, thresholds, runtime contract, numerical-verification report, and full validation-set quantization report. Its decision threshold is recalibrated on the validation split for the quantized logits; the SafeTensors checkpoint continues to use the canonical threshold in config.json. Use MODEL_BACKEND=onnx MODEL_PRECISION=int8 in the included FastAPI example. The conservative auto default retains PyTorch/SafeTensors on every device because INT8 speedups depend on the server CPU's instruction set and workload; benchmark before enabling it in production.

The measured macro-F1 change after INT8 threshold recalibration is +0.0009 (positive means a drop). See onnx/quantization_report.json for source-threshold results, logit differences, and local latency.

Mobile inference (ONNX Runtime Mobile)

The mobile/ directory contains an ARM-targeted, fixed-optimization ORT-format version of the validated dynamic-INT8 ONNX graph for ONNX Runtime Mobile on Android and iOS. It preserves dynamic batch and sequence inputs and uses the INT8 package's separately calibrated thresholds. Start with CPUExecutionProvider; accelerator providers require a separately converted/runtime-optimized graph and testing on each target device. mobile/required_operators_and_types.config can be passed to an ONNX Runtime minimal build so the application includes only the operators this graph needs.

This artifact is not proof of good phone performance. Model download size, peak memory, startup, latency, battery use, and thermal behavior must be measured on representative low- and high-end devices before release. The 97M model is the recommended mobile/edge candidate; the 311M model is primarily the highest-accuracy server option and its mobile graph is unusually large.

Private browser inference (WebGPU and WASM CPU)

The webgpu/ directory contains a verified, fixed-shape ONNX Runtime Web model for private, client-side inference. It uses FP32 linear weights and Q4 embedding weights, batch size 1, and the same saved per-label thresholds as the Transformers model. Load webgpu/webgpu_config.json first; it records the graph checksum, tokenizer artifacts, input contract, and validation report. Applications can execute the same graph with ONNX Runtime Web's WebGPU provider or its WASM CPU provider. WASM is more broadly compatible but can be substantially slower. A privacy-preserving fallback should try WASM locally and require explicit consent before switching to server inference.

Install the browser libraries with:

npm install @huggingface/transformers onnxruntime-web

Run inference inside a Web Worker so model loading does not block the page. This WebGPU example loads the pinned release and applies the saved per-label thresholds from webgpu_config.json:

import { AutoTokenizer } from "@huggingface/transformers";
import * as ort from "onnxruntime-web/webgpu";

ort.env.wasm.wasmPaths = "/ort/"; // Serve matching ORT Web helper files here.
const executionProviders = ["webgpu"];

const modelId = "NajahUniv/AraUni-Granite-97M-Intent-Classifier";
const revision = "v0.4.1";
const artifactRoot =
  `https://huggingface.co/${modelId}/resolve/${revision}/webgpu/`;
const config = await fetch(`${artifactRoot}webgpu_config.json`).then((response) => {
  if (!response.ok) throw new Error(`Could not load browser config: ${response.status}`);
  return response.json();
});
const tokenizer = await AutoTokenizer.from_pretrained(modelId, { revision });
const session = await ort.InferenceSession.create(
  `${artifactRoot}${config.model_file}`,
  { executionProviders, graphOptimizationLevel: "all" },
);

const text = "ما هي شروط القبول في الجامعة؟";
const encoded = await tokenizer(text, {
  padding: "max_length",
  truncation: true,
  max_length: config.max_length,
});
const feeds = Object.fromEntries(
  config.input_names.map((name) => [
    name,
    new ort.Tensor("int64", encoded[name].data, encoded[name].dims),
  ]),
);
const output = await session.run(feeds);
const logits = output[config.output_name].data;
const predictions = config.labels.map((label, index) => {
  const score = 1 / (1 + Math.exp(-Number(logits[index])));
  const threshold = Number(config.thresholds[label]);
  return { label, score, threshold, selected: score >= threshold };
});
console.table(predictions);

For private CPU execution, put the same shared inference code in a separate worker and change the runtime setup to:

import * as ort from "onnxruntime-web/wasm";

ort.env.wasm.wasmPaths = "/ort/";
ort.env.wasm.numThreads = 1; // Works without cross-origin isolation.
const executionProviders = ["wasm"];

Pass executionProviders to InferenceSession.create. Copy the matching ONNX Runtime Web .mjs and .wasm files from node_modules/onnxruntime-web/dist/ to the public /ort/ directory. Keep WebGPU and WASM in separate workers/bundles; do not assume that adding "wasm" after "webgpu" in one provider list provides a compatible fallback for every quantized operator. If WebGPU fails, start the WASM worker explicitly. Only switch to a server runtime after obtaining user consent.

This is multi-label classification: apply sigmoid independently to every logit, compare each score with that label's saved threshold, and allow zero, one, or several labels to be selected. The sigmoid scores are ranking/confidence signals and are not guaranteed to be calibrated probabilities.

Basic usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "NajahUniv/AraUni-Granite-97M-Intent-Classifier"
model_revision = "v0.4.1"
tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    revision=model_revision,
    trust_remote_code=True,
)
model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    revision=model_revision,
    trust_remote_code=True,
).eval()

text = "ما هي شروط القبول في الجامعة؟"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=model.config.max_length)
with torch.inference_mode():
    probabilities = torch.sigmoid(model(**inputs).logits[0])

result = {}
for index, probability in enumerate(probabilities.tolist()):
    label = model.config.id2label[index]
    threshold = model.config.thresholds[label]
    result[label] = {"probability": probability, "selected": probability >= threshold}

selected_labels = [label for label, value in result.items() if value["selected"]]
print(selected_labels)

A complete command-line example is included at examples/basic_inference.py:

python examples/basic_inference.py \
  --model-id NajahUniv/AraUni-Granite-97M-Intent-Classifier \
  --revision v0.4.1 \
  --backend auto \
  --precision auto \
  --text "ما هي شروط التسجيل؟"

<details> <summary>Complete basic inference example</summary>

"""Run multi-label inference with a published PyTorch or CPU INT8 ONNX model."""

from __future__ import annotations

import argparse
import json
from pathlib import Path

import numpy as np
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoConfig, AutoModelForSequenceClassification, AutoTokenizer

DEFAULT_MODEL_ID = "NajahUniv/AraUni-Granite-97M-Intent-Classifier"


def choose_device(requested: str) -> str:
    if requested != "auto":
        return requested
    if torch.cuda.is_available():
        return "cuda"
    if torch.backends.mps.is_available():
        return "mps"
    return "cpu"


def resolve_runtime(backend: str, precision: str, device: str) -> tuple[str, str]:
    resolved_backend = "pytorch" if backend == "auto" else backend
    resolved_precision = precision
    if precision == "auto":
        if resolved_backend == "onnx":
            resolved_precision = "int8"
        elif device == "cuda":
            resolved_precision = "bf16" if torch.cuda.is_bf16_supported() else "fp16"
        else:
            resolved_precision = "fp32"
    if resolved_backend == "onnx" and (device != "cpu" or resolved_precision != "int8"):
        raise ValueError("the published ONNX artifact supports CPU INT8 only")
    if resolved_backend == "pytorch" and resolved_precision == "int8":
        raise ValueError("precision=int8 requires backend=onnx")
    if resolved_backend == "pytorch" and resolved_precision in {"bf16", "fp16"} and device != "cuda":
        raise ValueError("the example enables bf16/fp16 only on CUDA")
    return resolved_backend, resolved_precision


def hub_or_local_file(model_id: str, filename: str, revision: str | None) -> str:
    local = Path(model_id) / filename
    if local.is_file():
        return str(local)
    return hf_hub_download(model_id, filename, revision=revision)


def main() -> None:
    parser = argparse.ArgumentParser(description=__doc__)
    parser.add_argument("--model-id", default=DEFAULT_MODEL_ID)
    parser.add_argument("--revision", help="Pin a release tag or reviewed commit SHA")
    parser.add_argument("--text", required=True)
    parser.add_argument("--device", default="auto", choices=("auto", "cpu", "mps", "cuda"))
    parser.add_argument("--backend", default="auto", choices=("auto", "pytorch", "onnx"))
    parser.add_argument(
        "--precision",
        default="auto",
        choices=("auto", "fp32", "bf16", "fp16", "int8"),
    )
    parser.add_argument("--top-k", type=int, default=5)
    args = parser.parse_args()

    device = "cpu" if args.backend == "onnx" and args.device == "auto" else choose_device(args.device)
    backend, precision = resolve_runtime(args.backend, args.precision, device)
    load_kwargs = {"revision": args.revision} if args.revision else {}
    tokenizer = AutoTokenizer.from_pretrained(
        args.model_id,
        trust_remote_code=True,
        **load_kwargs,
    )
    config = AutoConfig.from_pretrained(
        args.model_id,
        trust_remote_code=True,
        **load_kwargs,
    )
    thresholds = config.thresholds
    if backend == "onnx":
        import onnxruntime as ort

        onnx_config_path = hub_or_local_file(
            args.model_id,
            "onnx/onnx_config.json",
            args.revision,
        )
        onnx_config = json.loads(Path(onnx_config_path).read_text(encoding="utf-8"))
        thresholds = onnx_config["thresholds"]
        model_path = hub_or_local_file(
            args.model_id,
            "onnx/model_int8.onnx",
            args.revision,
        )
        session = ort.InferenceSession(model_path, providers=["CPUExecutionProvider"])
        encoded = tokenizer(
            args.text,
            return_tensors="np",
            truncation=True,
            max_length=config.max_length,
        )
        logits = session.run(
            ["logits"],
            {
                "input_ids": encoded["input_ids"].astype(np.int64),
                "attention_mask": encoded["attention_mask"].astype(np.int64),
            },
        )[0][0]
        probabilities = 1.0 / (1.0 + np.exp(-logits))
    else:
        dtype = {
            "fp32": torch.float32,
            "bf16": torch.bfloat16,
            "fp16": torch.float16,
        }[precision]
        model = AutoModelForSequenceClassification.from_pretrained(
            args.model_id,
            trust_remote_code=True,
            torch_dtype=dtype,
            **load_kwargs,
        ).to(device).eval()
        encoded = tokenizer(
            args.text,
            return_tensors="pt",
            truncation=True,
            max_length=model.config.max_length,
        ).to(device)
        with torch.inference_mode():
            probabilities = torch.sigmoid(model(**encoded).logits[0]).cpu().float().numpy()

    labels = [config.id2label[index] for index in range(config.num_labels)]
    ranked = sorted(
        (
            {
                "label": label,
                "probability": float(probabilities[index]),
                "threshold": float(thresholds[label]),
                "selected": float(probabilities[index]) >= float(thresholds[label]),
            }
            for index, label in enumerate(labels)
        ),
        key=lambda item: item["probability"],
        reverse=True,
    )
    print(
        json.dumps(
            {
                "text": args.text,
                "backend": backend,
                "precision": precision,
                "device": device,
                "selected_labels": [item["label"] for item in ranked if item["selected"]],
                "scores": ranked[: max(1, args.top_k)],
            },
            ensure_ascii=False,
            indent=2,
        )
    )


if __name__ == "__main__":
    main()

</details>

FastAPI hosting

The included service loads the model once, supports batching, uses MPS automatically on Apple Silicon, and optionally requires a bearer token:

pip install -r requirements.txt
MODEL_ID=NajahUniv/AraUni-Granite-97M-Intent-Classifier MODEL_REVISION=v0.4.1 MODEL_BACKEND=auto \
  MODEL_PRECISION=auto MODEL_API_KEY=change-me \
  uvicorn examples.fastapi_app:app --host 0.0.0.0 --port 8000

curl http://localhost:8000/classify \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer change-me' \
  -d '{"texts":["كيف يمكنني دفع الرسوم؟"],"top_k":5}'

In Swagger UI at http://localhost:8000/docs, click Authorize and enter only the MODEL_API_KEY value (change-me in the example). Swagger adds the Bearer prefix.

<details> <summary>Complete FastAPI server example</summary>

"""FastAPI service with automatic PyTorch or CPU INT8 ONNX inference."""

from __future__ import annotations

import hmac
import json
import os
import threading
from contextlib import asynccontextmanager
from pathlib import Path
from typing import Annotated

import numpy as np
import torch
from fastapi import Depends, FastAPI, HTTPException
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from huggingface_hub import hf_hub_download
from pydantic import BaseModel, Field
from transformers import AutoConfig, AutoModelForSequenceClassification, AutoTokenizer

DEFAULT_MODEL_ID = "NajahUniv/AraUni-Granite-97M-Intent-Classifier"
MODEL_ID = os.getenv("MODEL_ID", DEFAULT_MODEL_ID)
MODEL_REVISION = os.getenv("MODEL_REVISION")
MODEL_DEVICE = os.getenv("MODEL_DEVICE", "auto")
MODEL_BACKEND = os.getenv("MODEL_BACKEND", "auto")
MODEL_PRECISION = os.getenv("MODEL_PRECISION", "auto")
MODEL_API_KEY = os.getenv("MODEL_API_KEY")
MAX_BATCH_SIZE = int(os.getenv("MAX_BATCH_SIZE", "64"))


def choose_device() -> str:
    if MODEL_BACKEND == "onnx" and MODEL_DEVICE == "auto":
        return "cpu"
    if MODEL_DEVICE != "auto":
        return MODEL_DEVICE
    if torch.cuda.is_available():
        return "cuda"
    if torch.backends.mps.is_available():
        return "mps"
    return "cpu"


def resolve_runtime(device: str) -> tuple[str, str]:
    if MODEL_BACKEND not in {"auto", "pytorch", "onnx"}:
        raise ValueError("MODEL_BACKEND must be auto, pytorch, or onnx")
    if MODEL_PRECISION not in {"auto", "fp32", "bf16", "fp16", "int8"}:
        raise ValueError("MODEL_PRECISION must be auto, fp32, bf16, fp16, or int8")
    backend = "pytorch" if MODEL_BACKEND == "auto" else MODEL_BACKEND
    precision = MODEL_PRECISION
    if precision == "auto":
        if backend == "onnx":
            precision = "int8"
        elif device == "cuda":
            precision = "bf16" if torch.cuda.is_bf16_supported() else "fp16"
        else:
            precision = "fp32"
    if backend == "onnx" and (device != "cpu" or precision != "int8"):
        raise ValueError("the published ONNX artifact supports CPU INT8 only")
    if backend == "pytorch" and precision == "int8":
        raise ValueError("MODEL_PRECISION=int8 requires MODEL_BACKEND=onnx")
    if backend == "pytorch" and precision in {"bf16", "fp16"} and device != "cuda":
        raise ValueError("this example enables bf16/fp16 only on CUDA")
    return backend, precision


def hub_or_local_file(filename: str) -> str:
    local = Path(MODEL_ID) / filename
    if local.is_file():
        return str(local)
    return hf_hub_download(MODEL_ID, filename, revision=MODEL_REVISION)


class ClassifyRequest(BaseModel):
    texts: list[str] = Field(min_length=1)
    top_k: int = Field(default=5, ge=1)
    threshold: float | None = Field(default=None, gt=0, lt=1)


class ModelRuntime:
    def __init__(self) -> None:
        load_kwargs = {"revision": MODEL_REVISION} if MODEL_REVISION else {}
        self.tokenizer = AutoTokenizer.from_pretrained(
            MODEL_ID,
            trust_remote_code=True,
            **load_kwargs,
        )
        self.config = AutoConfig.from_pretrained(
            MODEL_ID,
            trust_remote_code=True,
            **load_kwargs,
        )
        self.device = choose_device()
        self.backend, self.precision = resolve_runtime(self.device)
        self.lock = threading.Lock()
        self.thresholds = self.config.thresholds
        self.model = None
        self.session = None
        if self.backend == "onnx":
            import onnxruntime as ort

            onnx_config = json.loads(
                Path(hub_or_local_file("onnx/onnx_config.json")).read_text(encoding="utf-8")
            )
            self.thresholds = onnx_config["thresholds"]
            self.session = ort.InferenceSession(
                hub_or_local_file("onnx/model_int8.onnx"),
                providers=["CPUExecutionProvider"],
            )
        else:
            dtype = {
                "fp32": torch.float32,
                "bf16": torch.bfloat16,
                "fp16": torch.float16,
            }[self.precision]
            self.model = AutoModelForSequenceClassification.from_pretrained(
                MODEL_ID,
                trust_remote_code=True,
                torch_dtype=dtype,
                **load_kwargs,
            ).to(self.device).eval()

    def probabilities(self, texts: list[str]) -> np.ndarray:
        if self.backend == "onnx":
            encoded = self.tokenizer(
                texts,
                return_tensors="np",
                padding=True,
                truncation=True,
                max_length=self.config.max_length,
            )
            with self.lock:
                logits = self.session.run(
                    ["logits"],
                    {
                        "input_ids": encoded["input_ids"].astype(np.int64),
                        "attention_mask": encoded["attention_mask"].astype(np.int64),
                    },
                )[0]
            return 1.0 / (1.0 + np.exp(-logits))
        encoded = self.tokenizer(
            texts,
            return_tensors="pt",
            padding=True,
            truncation=True,
            max_length=self.config.max_length,
        ).to(self.device)
        with self.lock, torch.inference_mode():
            return torch.sigmoid(self.model(**encoded).logits).cpu().float().numpy()

    def classify(self, request: ClassifyRequest) -> list[dict[str, object]]:
        if len(request.texts) > MAX_BATCH_SIZE:
            raise HTTPException(413, f"at most {MAX_BATCH_SIZE} texts are allowed per request")
        probabilities = self.probabilities(request.texts)
        labels = [self.config.id2label[index] for index in range(self.config.num_labels)]
        results = []
        for text, row in zip(request.texts, probabilities, strict=True):
            scores = []
            for index, label in enumerate(labels):
                threshold = (
                    request.threshold
                    if request.threshold is not None
                    else float(self.thresholds[label])
                )
                scores.append(
                    {
                        "label": label,
                        "probability": float(row[index]),
                        "threshold": threshold,
                        "selected": float(row[index]) >= threshold,
                    }
                )
            scores.sort(key=lambda item: item["probability"], reverse=True)
            results.append(
                {
                    "text": text,
                    "selected_labels": [item["label"] for item in scores if item["selected"]],
                    "scores": scores[: min(request.top_k, len(scores))],
                }
            )
        return results


runtime: ModelRuntime | None = None


@asynccontextmanager
async def lifespan(_: FastAPI):
    global runtime
    runtime = ModelRuntime()
    yield
    runtime = None


app = FastAPI(title="AraUni Multi-label Intent Classifier", lifespan=lifespan)
bearer_scheme = HTTPBearer(
    auto_error=False,
    scheme_name="BearerAuth",
    description="Enter the MODEL_API_KEY value. Swagger adds the 'Bearer' prefix.",
)


def authorize(
    credentials: Annotated[
        HTTPAuthorizationCredentials | None,
        Depends(bearer_scheme),
    ],
) -> None:
    if MODEL_API_KEY is None:
        return
    if (
        credentials is None
        or credentials.scheme.lower() != "bearer"
        or not hmac.compare_digest(credentials.credentials, MODEL_API_KEY)
    ):
        raise HTTPException(
            401,
            "invalid bearer token",
            headers={"WWW-Authenticate": "Bearer"},
        )


@app.get("/health")
def health() -> dict[str, object]:
    return {
        "status": "ok",
        "model_id": MODEL_ID,
        "device": runtime.device if runtime else None,
        "backend": runtime.backend if runtime else None,
        "precision": runtime.precision if runtime else None,
    }


@app.get("/labels", dependencies=[Depends(authorize)])
def labels() -> dict[int, str]:
    if runtime is None:
        raise HTTPException(503, "model is not ready")
    return dict(runtime.config.id2label)


@app.post("/classify", dependencies=[Depends(authorize)])
def classify(request: ClassifyRequest) -> list[dict[str, object]]:
    if runtime is None:
        raise HTTPException(503, "model is not ready")
    return runtime.classify(request)

</details>

For public production hosting, also add TLS, request-size/rate limits, monitoring, and a pinned MODEL_REVISION commit SHA. On macOS, use one Uvicorn worker so multiple processes do not each load a separate copy of the model into memory.

Labels

• 0: academic_calendar
• 1: academic_programs
• 2: admissions
• 3: campus_services
• 4: contact_and_location
• 5: courses_and_study_plans
• 6: exams_and_grades
• 7: general_university_information
• 8: graduation
• 9: library
• 10: news_and_events
• 11: out_of_scope
• 12: registration
• 13: research_and_postgraduate
• 14: scholarships_and_aid
• 15: staff_and_departments
• 16: student_services
• 17: technical_support
• 18: transfer_and_equivalency
• 19: tuition_and_payments

Intended use and limitations

The model is intended for routing Arabic university-chatbot questions within the label taxonomy above. It should not be treated as an authoritative source of admissions, academic, payment, or policy advice. Every example in the training, validation, and test splits is synthetic; none is a real-user query. Consequently, strong held-out scores measure performance only on the synthetic evaluation distribution and provide no guarantee of performance on real users' queries. Before any use, manually test representative queries from the intended users, languages, dialects, spelling patterns, and university context. For deployment, also evaluate on real, independently collected traffic and add monitoring plus human fallback/escalation. Inputs outside the training distribution can still receive confident scores. The sigmoid values are classification scores, not guaranteed calibrated probabilities.

Reproducibility and repository contents

• model.safetensors: complete encoder and classifier weights (the only weight copy)

• config.json: architecture, label mappings, pooling, maximum length, and thresholds

• tokenizer.json and tokenizer_config.json: tokenizer artifacts

• configuration_arauni.py and modeling_arauni.py: AutoClass code

• dataset_provenance.json, training_args.json, and metadata.json: provenance

• validation_metrics.json, test_metrics.json, and thresholds.json: evaluation artifacts

• benchmark_results.json: shared MPS/CPU speed comparison and environment metadata

• onnx/: validated dynamic-INT8 ONNX CPU graph and verification reports

• mobile/: ARM-targeted INT8 ORT graph, minimal-build operator config, and reports

• webgpu/: verified ONNX Runtime Web graph, tokenizer, thresholds, and reports

This model card reports the saved checkpoint artifacts; consult the linked base-model card for its pretraining details, license, intended uses, and limitations.

License

This model is released under the Apache License 2.0.