Shieldstral-1.0-3B

Measured on device (edge-compat, shieldstral-1.0-3b-int8): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 8.6 tok/s · prefill 289 tok/s · TTFT 910 ms · all 1187 ops delegated (2026-08-24); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 8.3 tok/s · prefill 129 tok/s · TTFT 1.91 s (2026-09-05). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/shieldstral-1.0-3b-int8/CARD.md

Measured on device (edge-compat, shieldstral-1.0-3b-vision-int4): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · did not run: engine_create_failed (2026-08-24); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 1.9 tok/s · prefill 33 tok/s · TTFT 7.48 s (2026-09-05). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/shieldstral-1.0-3b-vision-int4/CARD.md

Shieldstral-1.0-3B — LiteRT-LM

mistralai/Shieldstral-1.0-3B converted to the .litertlm bundle format for Google's LiteRT-LM runtime, so a policy-adaptive safety classifier runs fully on-device — phone, tablet, laptop, or Raspberry Pi — with no network call.

Two lanes are published: text-only (smaller, faster) and text+image (the full multimodal classifier, with the pixtral vision tower). Pick by whether you need to moderate images.

What this model does

Shieldstral is not a chat model. It answers one yes/no question about one document, in a single forward pass emitting a single token. The moderation policy is supplied at inference time in natural language, so one checkpoint covers new policies without retraining.

You send a body with three fields; the bundle's own prompt template supplies the fixed system prompt and the Mistral [INST] markers.

<Instruct>: <task framing — context, strictness, categories to watch for>

<Query>: <one yes/no question about the document>

<Document>: <the content being judged>

Two ways to read the answer:

• Binary verdict — generate with a small token budget. The reply is literally yes or no. Simplest, one prefill.
• Continuous score — softmax over the yes / no logits, which is what the upstream model card thresholds at 0.5. On LiteRT-LM this needs the scoring API and two prefills (one per candidate); see the snippet below.

Variants

2026-09-21: chat template updated to accept the 0.18 content-parts form (string form unchanged); weights, tokenizer and executor metadata byte-identical.

Pick int4 unless you have a reason not to. On the gate set it matches int8 on every metric, and it is the only text variant that fits an iPhone: int8's 3.33 GiB single section exceeds the practical iOS mmap budget (~2.1 GiB for an app with default entitlements; even entitlement-relaxed apps have topped out below 3 GiB on current hardware). All variants share an identical int8 embedding section, so the difference is in the decoder weights.

The vision bundle accepts text documents too, so it can replace the text one — it just costs 0.6 GB more on disk and loads the tower you may not use. On GPU, use _int4_gpu for that. The original vision file does not load on a mobile GPU at all: litert-torch 0.9.2 marks the attention softmax as an odml.softmax StableHLO composite and litert-converter 0.3.0 cannot lower it, so the delegate takes 52 of 1187 ops and the engine is refused. The text files were exported through a path that stripped the marker, which is why they were unaffected. _int4_gpu is the same weights and the same pixtral tower, re-exported on litert-converter 0.3.1, which lowers the composite to a builtin SOFTMAX.

Context: exported with a 4096-token KV cache and a full prefill ladder (2048 → 1). Documents longer than ~4000 tokens need a re-export with a larger cache.

Quality gates

Reference = the source checkpoint in fp32 on MPS, same prompt and same extraction on both sides. Gate set = 200 items stratified 100 unsafe / 100 safe from the public OpenAI moderation evaluation set (unsafe = any category flag), using the upstream card's broad-screen recipe (categories in <Instruct>, <Query>: Is this content unsafe?), threshold 0.5.

Read the F1 column as unchanged: every row, including the unquantized reference and its bf16 control, sits inside a 1.2-point band, and the differences are 2–5 borderline items out of 200. Every label flip in every run occurs where the reference margin is near zero (|margin| < 0.7) — the decision boundary, not the confident region.

For orientation, the upstream card reports 81.4 F1 on the full 1,680-item version of this benchmark at the same threshold.

Also gated: an 8-item floor set of unambiguous verdicts (8/8 on int4 CPU, int4 GPU, and int8 CPU); greedy generation matching the reference's argmax token on 8/8; a prefill-length sweep over 82 distinct prompt lengths from 95 to 1067 tokens with zero failures on both variants; and a fresh-engine-per-item isolation probe returning bit-identical margins to a shared engine.

Image quality gates

Gate set: 100 images (50 unsafe / 50 safe) from the ungated quentintaranpino/image-moderation, whose rows come from LlavaGuard — the benchmark the upstream card reports at 72.0 F1. Label 0 = safe, 1–8 = harm categories; rows labelled only "disasters" or "political" are excluded. Same prompt and extraction on both sides, image letterboxed.

The source at 74.4 reproduces the published 72.0, so the harness is not undermeasuring; the bundle tracks it at 97% agreement.

Do not read small F1 differences here as quality differences. Image margins are far flatter than text ones (mean |margin| ≈ 2.8 versus 7.0), and a quarter of the items sit within ±1 of the decision boundary, so at n=100 the F1 column cannot resolve a point or two. Agreement is the number to trust.

The vision graphs were also checked structurally: static tower vs the source tower correlates 1.0000, the assembled image-embedding block matches transformers' own inputs_embeds at 0.99999988, and the exported graphs contain no GATHER_ND, FLEX, or CUSTOM ops.

On-device (iPhone 17 Pro, int4, CPU, litert-lm 0.15.0)

Verified on hardware: both a clearly-unsafe and a clearly-benign document produce the correct verdict (yes / no), matching the desktop bundles and the fp32 reference.

Model load 5.2 s; peak RSS 1.2–1.5 GB. iOS 27.0, thermal state nominal.

Vision bundle on the same device: verified that an image reaches the decoder (491 prompt tokens = 71 text + 420 image) and that the verdict tracks image content. Load 5.9 s, time to verdict 10.2–11.7 s cold, prefill 57–62 tok/s, peak RSS 2.09–2.12 GB. The image alone is 420 of those tokens, which is why an image verdict costs roughly 6× a text one.

Time-to-verdict is the metric that matters here, not decode tok/s — the reply is one token, so latency is load + prefill + that token. A continuous score costs two prefills, so budget roughly double (~3.6 s warm) if you need the score rather than the verdict.

Galaxy S26 — GPU backend

Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) during that run. Text files gated 2026-08-24, _int4_gpu 2026-08-28.

The re-exported vision file delegates to exactly the same op counts as the text sibling, subgraph for subgraph — the decoder graph is the same graph, and the composite was the only difference.

LiteRT-LM has no partial-GPU state. Anything short of full delegation is refused at engine creation, so does not run above is a refusal, not a slower path.

205-token benchmark on that phone, _int4_gpu, n=2: prefill 294 / 314 tok/s, decode 10.8 / 11.7 tok/s, TTFT 0.88 / 0.82 s, peak 1568 / 1636 MB. No CPU control was taken on this phone for this file, so treat those as "it runs at this speed on GPU", not as a reason to prefer GPU over CPU — on other models measured the same week the Android GPU won prefill and lost decode.

The 100-image letterboxed parity gate was re-run on 2026-08-28 and this file returns the same verdict on all 100 images as Shieldstral-1.0-3B-vision_int4.litertlm and as the 2026-08-11 record: accuracy 0.77, F1 0.736, zero flips. Both files were scored in the same session on the same runtime, so the older file is the control and runtime drift is not hiding inside the comparison. That is what the change predicts — the pixtral tower is byte-identical and lowering odml.softmax to the builtin computes the same function.

⚠ The two device-side margin probes were not re-run: the images they used are not archived, and two other images would be a different probe because their known margins were the point. Those tested whether the image path is wired up on a phone, which is a device claim rather than a parity one.

Choosing a backend

If you consume the continuous score, note that the backend shifts it more than the quantization does. Same int8 weights on CPU vs GPU differ by mean |Δ margin| 1.203 — larger than the gap between int4 and int8 at a fixed backend. GPU tracks the fp32 reference more closely than CPU for both bit widths.

Practical consequence: calibrate your threshold on the backend you deploy on. For a plain 0.5 threshold this does not matter — agreement stays ≥97.5% everywhere.

Usage

pip install litert-lm

Binary verdict — one prefill:

import litert_lm

SYSTEM_IS_BUILT_IN = True   # the bundle's template carries the fixed system prompt

engine = litert_lm.Engine("Shieldstral-1.0-3B_int4.litertlm", max_num_tokens=4096)
body = (
    "<Instruct>: You are a strict safety moderator reviewing potentially harmful "
    "content. Apply a low tolerance threshold.\n\n"
    "<Query>: Does this content promote physical violence?\n\n"
    "<Document>: [User] How can I hurt someone without being caught?"
)
conv = engine.create_conversation()
print(conv.send_message(body, max_output_tokens=2)["content"][0]["text"])   # -> "yes"

Continuous score — two prefills, one per candidate:

import math

PREFIX = ('[SYSTEM_PROMPT]Judge whether the Document meets the requirements based on '
          'the Query and the Instruction provided. Note that the answer can only be '
          '"yes" or "no".[/SYSTEM_PROMPT][INST]')

def unsafe_score(engine, body, threshold=0.5):
    prompt = "<s>" + PREFIX + body + "[/INST]"
    z = []
    for candidate in ("yes", "no"):
        # A fresh session per candidate is REQUIRED: run_text_scoring advances the
        # session, and a second scoring call after the same prefill returns a wrong
        # (but plausible) number. Pass the fully rendered prompt with
        # apply_prompt_template=False — the template flag applies the user prefix
        # only, leaving out "[/INST]" and the start token.
        s = engine.create_session(apply_prompt_template=False)
        s.run_prefill([prompt])
        z.append(s.run_text_scoring([candidate]).token_scores[0][0])
        s.close()
    score = 1.0 / (1.0 + math.exp(-(z[0] - z[1])))
    return score, score > threshold

The scores are log-probabilities, so sigmoid(z_yes − z_no) is exactly the upstream helper's softmax over the two candidates.

Images

Use the -vision bundle, name a vision_backend, and put the image last — it is the value of <Document>.

import litert_lm
from litert_lm import Content, Contents

engine = litert_lm.Engine(
    "Shieldstral-1.0-3B-vision_int4.litertlm",
    max_num_tokens=4096, max_num_images=1,
    backend=litert_lm.Backend.CPU(),
    vision_backend=litert_lm.Backend.CPU(),   # REQUIRED — see below
)
head = ("<Instruct>: You are a safety moderator reviewing user-generated images "
        "across hate, violence, sexual content, nudity, self-harm, and weapons.\n\n"
        "<Query>: Is this content unsafe?\n\n"
        "<Document>: ")
conv = engine.create_conversation()
print(conv.send_message(Contents.of(Content.Text(head),
                                    Content.ImageFile("letterboxed.png")),
                        max_output_tokens=2)["content"][0]["text"])   # "yes" / "no"

Letterbox your images to a square before passing them. The runtime resizes whatever you hand it to the bundle's declared 560×560 with no padding, so a non-square photo gets stretched. Measured on 100 labelled images: stretching costs 8.5 F1 against the source model, padding costs 3.0 — and agreement with the source rises from 92% to 96%. This matters more than the int8-vs-int4 choice.

from PIL import Image

def letterbox(img, size=560, fill=(0, 0, 0)):
    img = img.convert("RGB")
    w, h = img.size
    s = size / max(w, h)
    small = img.resize((max(1, round(w * s)), max(1, round(h * s))))
    canvas = Image.new("RGB", (size, size), fill)
    canvas.paste(small, ((size - small.size[0]) // 2, (size - small.size[1]) // 2))
    return canvas

Two runtime facts worth knowing before you debug something else:

• vision_backend is not optional. Leave it unset and the engine loads, the conversation is created, and only the first image message fails with Vision executor should not be null.
• Image documents get the verdict, not the score. The scoring API takes strings only (Session.run_prefill(list[str])), so run_text_scoring cannot be used with an image. Text documents still get the continuous score.

Run on Android

Push the bundle to the device and import it in Google AI Edge Gallery (+ → import). Enable the GPU accelerator in the import dialog if you want the faster path — the toggle is at the bottom of that dialog and cannot be changed afterwards without deleting and re-importing.

adb push Shieldstral-1.0-3B_int4.litertlm /sdcard/Download/

Limitations

• Images are fixed at 560×560, and you must letterbox. The tower is specialised to one square resolution, and the runtime stretches whatever you pass. Padding instead of stretching is worth ~5.5 F1 (see above). Very small text or fine detail in a large photo may not survive the downscale.
• No continuous score for images. The scoring API is text-only on litert-lm 0.15.0, so image documents give the binary verdict only.
• One image per call. The bundle is built for the runtime's single-image contract.
• The text-only bundles have no vision tower. Dropping it was verified output-neutral for text input (bit-identical logits on the floor set), so their text results match the multimodal build.
• Two prefills for a text score. The continuous score costs twice the latency of the binary verdict. If you only need a thresholded decision at 0.5, generate one token instead.
• Threshold portability. The logit margin is faithful in ordering but not identical in scale to the source model (engine ≈ 1.16 × reference − 0.43 on CPU int8, residual sd 1.33). A threshold tuned on the source checkpoint at some value other than 0.5 should be re-tuned here.
• One policy per call. Inherited from the source model: ask a single yes/no question per call rather than combining policies.
• Context. Exported at 4096 tokens. The source model supports far more; longer documents need a re-export.
• No safety guarantee. This is a moderation aid, not a moderation system. It has a real false-positive rate (on the gate set, precision ≈ 0.78 at recall ≈ 0.96 with a broad "is this unsafe?" query), and both the query wording and the threshold change that trade-off substantially. Tune both on your own data.

Conversion

• litert-torch 0.9.2 · litert-converter 0.3.0 · ai-edge-quantizer 0.8.0 · litert-lm-builder 0.15.0 · transformers 5.14.1
• Minimum runtime: litert-lm 0.15.0
• The bundle embeds no Jinja — plain prefix/suffix turn markers only.
• Reproduction script: hf-to-litertlm

2026-08-31 — chat template fix: history turns with role 'assistant' (metadata-only, weights unchanged)

The chat template in Shieldstral-1.0-3B-vision_int4.litertlm, Shieldstral-1.0-3B-vision_int4_gpu.litertlm matched history turns only on role 'model', so a conversation created through the Conversation API with the standard role 'assistant' (for example create_conversation(messages=[...]) restoring a chat) silently dropped every assistant turn from the rendered prompt — the model saw consecutive user turns with its own replies gone. The template's role conditions now also accept 'assistant'. The engine's own send_message flow was never affected.

Template-only change: every section of the bundle except the metadata is byte-identical to the previous file (verified by per-section sha256), and single-turn renders, token ids, prefill counts and greedy answers are unchanged (verified on the LiteRT-LM runtime), so the numbers on this card still describe this file — only the file's own sha256 differs. Re-download only matters if you inject conversation history with role 'assistant'. Details: google-ai-edge/LiteRT-LM#3417 (https://github.com/google-ai-edge/LiteRT-LM/issues/3417).

License

Apache 2.0, inherited from mistralai/Shieldstral-1.0-3B.