decider-0.8b-GGUF

decider-0.8b: typed decisions with calibrated probabilities in one forward pass

The smallest decider: a language model that does not generate text. It reads a state (a string or any JSON value) and a set of typed questions and returns a probability distribution for every question from a single forward pass: Choice (2-255 options, optionally described), Score (2-10 described levels), Noul (probability of yes). No decoding, no parsing, no output outside the options you defined. Same code, same wire format (POST /v1/systemone, TypeSafe Jev's format), same training recipe as the 2B: one epoch of scripts/train.sh full from Qwen/Qwen3.5-0.8B-Base over the full mixture (1.47M examples, 455M tokens, 4.5 h on one GH200).

Contents: The decider family · Usage · How it compares with the 2B · Limitations · Changelog

The decider family

All six repositories share one interface (decider.infer.Decider, POST /v1/systemone in TypeSafe's format) and one readout: the letter logits at an answer slot, softmaxed over the options. Pick by size and input.

Code, data registry, training scripts, the changelog and the per-version history: https://github.com/Mapika/decider.

Usage

# pip install git+https://github.com/Mapika/decider
from decider.infer import Decider
d = Decider("Mapika/decider-0.8b")
d.system_one(
    {"ticket": "I was charged twice for order A-104. Please refund the duplicate."},
    {"team": {"type": "choice", "instructions": "Which team should handle this?",
              "criteria": {"billing": "Charges, invoices, refunds", "technical": "Bugs, outages", "other": None}},
     "refund_requested": {"type": "noul", "instructions": "Does the customer ask for a refund?"},
     "frustration": {"type": "score", "instructions": "How frustrated is the customer?", "criteria": ["calm", "frustrated", "very frustrated"]}})

decide_batch, schema (the cached question set), decider.serve and the TypeSafe request shape work as in the decider-2b card; decider/ in this repository is the inference subset of the GitHub package.

How it compares with the 2B

Same 93 public tasks, same protocol, one temperature fitted on in-task data (it came out at 1.03 for both: the recipe calibrates at every size). "Held-out" means no example of that dataset was trained on.

What the smaller model gives up is knowledge, not the decision format: the largest drops are TruthfulQA (0.41 vs 0.55), OpenBookQA (0.67 vs 0.81), HellaSwag (0.76 vs 0.88) and ARC (0.74 vs 0.86), and wide label sets that need fine distinctions (TREC-fine). Routing, classification, yes/no judgments and JSON lookups on short states are within one to four points of the 2B. It is a weaker player: Pong and Breakout stay at the scripted teacher's level, CliffWalking fails (it walks off the cliff), held-out Freeway scores 0. The regression set runs about 1.5x faster than on the 2B; bf16 weights are 1.5 GB.

Limitations

Those of decider-2b, more so: a small model without reasoning; English only; rules written into the question ("fill if empty, otherwise skip") are not followed reliably, so state the decision as a plain question with described options; knowledge-heavy multiple choice is close to the base model; calibration is measured on public datasets and teacher-labelled probes, not on your traffic. The teacher-written training data comes from Qwen3.5-27B and carries its biases.

Changelog

Every decider release is listed in docs/CHANGELOG.md of the GitHub repository. Code and the training recipe: https://github.com/Mapika/decider.

GGUF

CPU inference is available through dohnuts.cpp, a native C++ port on llama.cpp. It runs the model on CPU, reads the option-letter logits at each Answer: ( slot from the LM head, and serves the same POST /v1/systemone wire format. No GPU or Python runtime is needed.

This conversion is verifiable and structural; it has not been benchmarked against the PyTorch reference. Q8_0 is the conservative default.

git clone https://github.com/DreamBlooms/dohnuts.cpp
cd dohnuts.cpp
git submodule update --init --depth 1
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON
cmake --build build -j --target dohnuts-cli

build/dohnuts-cli --server --port 8080 --model decider-0.8b-q8_0.gguf --metadata decider.json

Ask one state several questions (same request shape as the PyTorch server):

curl http://127.0.0.1:8080/v1/systemone -H 'Content-Type: application/json' \
  -d '{"state":{"ticket":{"messages":[{"from":"customer","text":"I was charged twice. Please refund the duplicate."}]}},
       "questions":{"department":{"type":"choice","instructions":"Which team should handle this?",
         "criteria":{"billing":null,"technical support":null,"sales":null}},
         "refund_requested":{"type":"noul","instructions":"Is a refund requested?"}}}'

The answer keeps the core fields (type, choice, probabilities, noul, confidence) and adds the model's native confidence and certainty under native, along with legend / level_fit / fit_mass for isolated Score levels.

Rebuild the file from the upstream weights with scripts/build_decider_gguf.sh (convert with --no-mtp; the base config declares an MTP layer but ships no MTP weights). No retraining or merging is involved.