decider-0.8b-executorch-mlx

decider-0.8b: compact ExecuTorch MLX export

Converted from Mapika/decider-0.8b, checkpoint revision 1ea54127d3bd52f6d753d9257b32a6380b873907. Original model, training, typed-decision interface and calibration: Mark Marosi / Mapika, based on Qwen3.5. No additional training or fine-tuning was performed for this export.

Execution changes and credits

FP16 weights/activations, FP32 recurrent state, fused projections/normalization, padding removal, and bounded parallel suffix recurrence. Prefixes may be shared within a request; no answers or state are cached across requests.

Export source: Decider MLX fork. PyTorch/ExecuTorch, Apple MLX, Qwen and Hugging Face Transformers supply the underlying implementation and runtime components. No upstream endorsement is implied. The Apache-2.0 license is included. These are export/runtime changes, not a newly trained model.

Download and call

Designed for local inference on Apple Silicon using the ExecuTorch MLX backend. Install the MLX runtime and fork, then:

from decider.mlx_backend import MLXDecider

model = MLXDecider.from_pretrained("edbordin-linktree/decider-0.8b-executorch-mlx")
result = model(
    {"ticket": "I was charged twice. Please refund the duplicate."},
    {"team": {"type": "choice", "instructions": "Which team should handle this?",
              "criteria": {"billing": "Charges, invoices and refunds",
                           "technical": "Bugs and technical problems"}}},
)
print(result["answers"])

Or start the compatible local HTTP server:

python -m decider.serve --backend mlx --model edbordin-linktree/decider-0.8b-executorch-mlx

The loader downloads and caches the complete bundle automatically, including calibration and tokenizer assets. Set revision="COMMIT_HASH" in Python or --revision COMMIT_HASH on the server to pin a release. After the first download, Python's local_files_only=True loads the cached bundle offline. This is an ExecuTorch artifact, not an mlx-lm or Transformers checkpoint.

Only load trusted artifacts. Verify files against SHA256SUMS. Provenance and the binary checksum are in provenance.json. Tested: Python 3.12, Torch 2.14.0, ExecuTorch 1.5.0, Transformers 5.17.0, M4 Pro Mac with 48 GB memory. Other runtime versions/hardware and operation without full Xcode are not yet validated.

Limits

• Maximum 575 rendered tokens per row, subject to separate prefix and suffix bounds of 512 and 63 tokens.
• Up to 6 rows per forward; the adapter splits larger groups.
• No larger reference fallback: unsupported shapes fail. No vision or 32k context support in this compact artifact.
• Inherits the base model's limitations, including imperfect conditional-rule following. Small numerical parity checks are not a general accuracy benchmark.
• Run one model at a time. Export/parity tests use much more memory than serving. A 48 GB test machine does not establish compatibility with every smaller Mac.

See the upstream model card for training details and evaluations; those results belong to the model authors, not this conversion.