zico

zico

zico is a 4B decision model. You give it a state (a message, a document, a JSON object, an image) and a set of typed questions. It answers every question with a calibrated probability distribution over the options you listed — in one forward pass, without generating any text.

// request                                                   // response
{ "state": { "ticket": { "body": "Charged twice, refund me    { "answers": {
             or I cancel." } },                                   "wants_refund": { "type": "noul", "noul": 0.89 },
  "questions": {                                                  "team": { "type": "choice", "choice": "billing",
    "wants_refund": { "type": "noul",                                       "probabilities": { "billing": 0.94, "technical": 0.02, … },
        "instructions": "The customer asks for a refund." },                "confidence": 0.81 },
    "team": { "type": "choice",                                   "frustration": { "type": "score", "score": 2.26,
        "instructions": "Which team should handle this?",                   "probabilities": { "0": 0.04, "1": 0.08, "2": 0.54, … } } },
        "criteria": { "billing": "Charges, refunds",            "usage": { "input_tokens": 240, "output_tokens": 0 } }
                      "technical": "Bugs, outages", … } },
    "frustration": { "type": "score",
        "criteria": ["Calm", "Annoyed", "Frustrated", "Angry", "Furious"] } } }

It is meant for the many small decisions inside software — routing, triage, moderation, guardrails, tool selection, "is this answer grounded?", document and image sorting — where you want a fast, cheap, typed answer with a probability you can threshold, not a paragraph to parse.

• Type-safe by construction. The output is a softmax over exactly the options in your request. An answer outside your schema cannot happen, so there is nothing to parse, retry or validate.
• Calibrated. Trained with strictly proper scoring rules; on held-out tasks its stated confidence tracks its accuracy (expected calibration error ≈ 0.04–0.06, see below). "0.9" means roughly nine in ten.
• Many questions, one pass. The state is encoded once and every question branches off it, so ten questions cost little more than one. Answers are identical whether questions are asked together or alone.
• Text and images. Images can sit anywhere inside the state, next to text fields.
• Fast. ≈110 ms for 1–16 questions about a 500-token state on an H100 (64 questions: ≈260 ms); ≈300 ms for a 4-question request on an M4 Max laptop with the included MLX backend.

Quick start

pip install -U "huggingface_hub[cli]"
hf download AmeenAhmed2/zico --local-dir zico && cd zico
python -m venv .venv && source .venv/bin/activate
pip install -e ".[serve]"            # Apple silicon: pip install -e ".[serve,mlx]"
python scripts/infer.py . examples/request.json

On Apple silicon scripts/infer.py (PyTorch) works but is slow — there is no fused Gated-DeltaNet kernel for MPS, so a request takes ≈1.7 s; the server below uses the MLX backend there and answers in ≈0.3 s.

The repository is the checkpoint and the code that runs it (zico/, a small Python package). The model cannot be loaded with transformers.AutoModel alone: the language backbone is a standard Qwen3.5 checkpoint, but the decision head and the input format are zico's own.

import torch
from zico import DecisionModel, Choice, Score, Noul

model, renderer, _ = DecisionModel.load(".", device="cuda", dtype=torch.bfloat16)   # or "mps" / "cpu"
model.autocast_dtype = torch.bfloat16

answers = model.answer(
    renderer,
    state={"message": "hey can u move my 3pm with dr patel to next week, mornings are better"},
    questions={
        "intent": Choice("What does the patient want?", {"book": None, "reschedule": None, "cancel": None, "billing": None, "other": None}),
        "time_preference": Noul("The patient states a preferred time of day."),
        "urgency": Score("How urgent is this?", ["Whenever", "This week", "Today", "Emergency"]),
    },
)
print(answers["answers"]["intent"])   # {'type': 'choice', 'choice': 'reschedule', 'probabilities': {...}, 'confidence': ...}

An image is a JSON object inside the state — {"type": "image", "data": "<base64>"}, optionally with "detail": "low" | "high" (64 to 1,024 image tokens; default ≤ 256):

import base64
photo = {"type": "image", "data": base64.b64encode(open("scan.png", "rb").read()).decode()}
model.answer(renderer, {"scan": photo}, {"kind": Choice("What type of document is this?", {"invoice": None, "letter": None, "form": None, "resume": None})})

HTTP server and test page

ZICO_CKPT=. scripts/serve.sh start        # also: stop | restart | status | log | key-path

This starts a small FastAPI server on http://127.0.0.1:8787 (on Apple silicon it uses the MLX backend automatically):

• POST /v1/systemone with Authorization: Bearer <key> takes the JSON request shown at the top (plus "model": "zico") and returns the response shown there. A key is generated on first start; scripts/serve.sh key-path says where it is.
• GET / is a test page: write a state, drop in an image, build questions, and watch the probability bars. "Live" mode re-runs on every edit and shows how far each probability moved.

The request and response shapes follow the System One wire format that TypeSafe AI documents publicly for its API, so client code written for that format can be pointed at this server (ZICO_MODEL_ALIASES adds any fixed model name such a client sends). See "Relation to other work" below.

How it works

row    =  ### State\n{state}\n\n   ### Choice\n{instructions}\n- billing: Charges…<OPT>\n- technical: Bugs…<OPT>\nAnswer:<ANS>
          └──── shared prefix ────┘ └──────────────────────────── one question ──────────────────────────────────────┘

logit_i = q(h[<ANS>]) · k(h[<OPT>_i]) / √d  +  u(h[<OPT>_i])          p = softmax over this question's own options

• Backbone. Qwen/Qwen3.5-4B-Base (a hybrid of Gated-DeltaNet and attention layers), fully fine-tuned, language-model head removed. It is only ever used for prefill: there is no decoding loop.
• Slots, not tokens. Each option ends in an <OPT> slot and each question in an <ANS> slot. Their input embeddings come from a learned 4-row table, so the tokenizer and embedding matrix are untouched.
• Pointer head. The hidden state at <ANS> is scored against the hidden state at every <OPT>; the softmax runs over exactly those positions. Choice → arg-max key, Score → Σ i·pᵢ, Noul → p(true), confidence = 1 − H(p)/log K. The head has 1.3M parameters.
• One state, many questions. Training uses one (state + question) row per question. At inference the state is prefilled once and its cache — attention KV and the recurrent / convolution state of the DeltaNet layers — is forked across the questions, which run as one batch. Because the backbone is causal both compute the same function; the test-suite asserts it.
• Images enter as embeddings from Qwen3.5's own vision tower (frozen, shipped in vision/), placed where the image sits in the state's JSON, with Qwen's 3-part rotary positions. They are part of the shared prefix, so an image is encoded once for all questions.
• Loss. Cross-entropy + 0.5 · Brier score, plus the ranked probability score for Score questions: all strictly proper scoring rules, so the optimum is the true conditional distribution — accuracy and calibration are the same objective. A per-type temperature fitted on validation data is stored in the checkpoint (1.09 / 1.10 / 1.23).

Training

1. Text pass — 1 epoch over ≈40 public classification, NLI, reading-comprehension, multiple-choice, intent, moderation, retrieval-relevance and tool-selection datasets plus ≈300 Super-NaturalInstructions tasks, all recast as typed questions. To make the model read the schema rather than memorise label sets, records randomise option order, option subsets, the presence of descriptions, instruction wording, whether the state is text or JSON, and add "none of the above" cases. Whole datasets and whole instruction tasks are held out, and training text is de-duplicated against every held-out set. ≈2.5 h on one H100 (bf16, 8-bit AdamW, ≈8k tokens/s).
2. Image pass — 1 epoch over 90k images from 12 public image-classification sets (documents, objects, fine-grained species / cars / aircraft, textures, satellite and aerial scenes, traffic signs, product attributes), vision tower frozen. To stop the text skills drifting, 80k text rows were replayed with the previous model's own probabilities as soft targets — not the gold labels. Replaying hard labels made the model markedly overconfident on tasks it had never seen (ECE 0.045 → 0.131); anchoring to its own distributions avoided that (0.055) while keeping every image gain. ≈85 min on one H100.

Training and data-building code is in scripts/ (build_data.py, build_image_data.py, make_soft_targets.py, train.py, eval.py).

Evaluation

All numbers are this checkpoint, bf16, raw probabilities unless stated. "Held out" means the dataset (or the whole instruction task) never appeared in training, so the label set and the wording are new to the model. Sample sizes are small (150–500 per set; 5.5k for the unseen-task pool): differences under ≈3 points are noise.

Text, held-out (zero-shot)

Text, in-distribution test split: 86.2 % over 37 sources (3,000 rows). Examples: Banking77 96.7 %, CLINC-150 with out-of-scope 95.3 %, AG News 93.0 %, MNLI 92.0 %, BoolQ 92.0 %, VitaminC fact-checking 92.9 %, SQuAD-v2 answerability 88.0 %, jailbreak / prompt-injection detection 100 % (small sets), SST-5 67.3 %, MS MARCO passage selection 57.1 %. By question type over the whole test split (text and images): Choice 90.0 %, Noul 92.3 %, Score 68.5 % exact level.

Images, held-out sets (zero-shot: never trained on, full label set in every question)

Images, test splits of the training sets. As in training, a question lists either the full label set or a random subset that contains the right answer, and about a quarter are "the image shows X" Noul questions — so these are easier than standard full-label benchmarks and are not comparable to published leaderboard numbers. 150–300 questions per set.

Calibration. Over all 9.9k held-out rows (text and images): ECE 0.055 raw, 0.041 with the stored temperatures. On the in-distribution test split: 0.011 raw. Full per-source results are in eval.json.

Consistency. Forked (batched) answers equal single-question answers to ≈1e-6 in fp32 (6e-5 in bf16 on an H100). The MLX backend matches PyTorch: 388 probabilities from real requests differ by 0.0003 on average, with the same top answer every time.

Limitations

• One forward pass, no reasoning. Weak at counting, arithmetic, dates, multi-hop logic and double negatives. If a question needs a chain of thought, this is the wrong tool.
• It classifies; it does not extract or generate. No spans, no free text, no numbers beyond a 2–10 level rubric (and on rubrics the exact level is right about two times in three; the expected value is the more useful output).
• Trained on rows up to ≈1k tokens. The server accepts states up to 8k tokens, but accuracy and calibration on long inputs are untested.
• Mostly English. The backbone is multilingual; the fine-tuning data is almost entirely English.
• Fine-grained subjective labels are hard (emotion 58 %, 5-star sentiment 67 %), and there the model is overconfident. Calibration is a property of the distributions it was tested on; check it on your own data before relying on a threshold.
• Images: single still images, classification-style questions. It is not an OCR engine and was not trained on charts, screenshots, multi-page documents or video. Small text inside a photo may not be read.
• Safety-related questions (toxicity, jailbreaks, prompt injection) were trained on small public sets. Treat the output as one signal, not as a complete moderation or security system, and keep a human in the loop for decisions that matter to people.
• The training data carries the usual biases of public web, review and social-media corpora.

Licence and training data

• Weights: CC-BY-NC-4.0 (non-commercial). The base model is Apache-2.0, but some of the training data is licensed for non-commercial or research use only — notably MS MARCO and ToxicChat (text), and ImageNet, FGVC-Aircraft, CUB-200, Stanford Cars, SUN397 and RVL-CDIP (images). The cautious reading is that a model trained on them should not be used commercially. scripts/build_data.py --exclude-nc builds the text set without the two non-commercial text sources if you want to train a variant.
• Code (zico/, scripts/, tests/): Apache-2.0.
• vision/ is the unmodified vision tower of Qwen/Qwen3.5-4B-Base (Apache-2.0, © Alibaba Cloud), redistributed so the checkpoint is self-contained. See NOTICE.
• The datasets are listed in this card's metadata. They remain under their own licences; none of their content is redistributed here.

Relation to other work

• Built on Qwen3.5-4B-Base by the Qwen team; the Apple-silicon backend builds on mlx-lm.
• The task format — a state plus typed Choice / Score / Noul questions answered with probabilities — and the HTTP wire format follow the System One API that TypeSafe AI documents publicly. zico is an independent, from-scratch project: it is not affiliated with or endorsed by TypeSafe AI, shares no code or weights with their models, and no output of their models was used to train or tune it. The architecture described above is this project's own design and says nothing about how theirs works.

Citation

@misc{zico2026,
  title  = {zico: a calibrated, typed decision model in a single forward pass},
  author = {AmeenAhmed2},
  year   = {2026},
  url    = {https://huggingface.co/AmeenAhmed2/zico}
}