Eikos-4B-MLX-8bit

Eikos-4B-MLX-8bit

MLX 8-bit build of Eikos-4B: MLX weights quantized to 8 bits (group size 64; mlx_lm convert -q --q-bits 8 --q-group-size 64) for Apple Silicon. The conversion keeps the text model only (no vision tower, no MTP weights), and the embeddings and LM head are quantized like the other layers. The prompt format, the letter readout and the calibration (calib.json, T = 1) are the same as the bf16 model.

Eikos answers typed decisions (yes/no, one of N options, ordinal scores) about a given state in one forward pass, with a calibrated probability for every option. See the Eikos-4B card for what the model does, how it was trained, the full evaluation and its limitations.

Use

pip install mlx mlx-lm
hf download caiovicentino1/Eikos-4B-MLX-8bit --local-dir Eikos-4B-MLX-8bit
python Eikos-4B-MLX-8bit/mlx_decide.py Eikos-4B-MLX-8bit          # runs a few demo decisions

import sys; sys.path.insert(0, "Eikos-4B-MLX-8bit")
from decision_core import options_of
from mlx_decide import MLXDecider

d = MLXDecider("Eikos-4B-MLX-8bit")
q = {"type": "noul", "instructions": "Under rule 4.2, can this order be executed as submitted?",
     "criteria": {"true": "complies with rule 4.2", "false": "breaches rule 4.2"}}
probs, n_tokens = d.dist("Order ticket #A-2231 ... Rule 4.2: ... Approvals on file: none.", q, options_of(q))

mlx_decide.py uses the same letter readout, prompt and calibration as the GPU builds. It also answers several questions over one state in a single cached batch (dist_many_cached).

Text only: this build has no vision weights, so it does not read images. Images need a GPU build with vLLM and serve v1.3: see Images.

Validation against bf16

Same 7,371 items for both builds (7 suites, never used in training), MLX run with its CUDA backend on Linux. The release gate was fixed before looking at results: accuracy within 1 point of bf16, ECE within 0.01, and at least 97% of answers unchanged.

It passes our release gate.

License

MIT for our contributions (LICENSE). The base model, Qwen3.5-4B, is Apache-2.0 (LICENSE-Qwen); attributions are in NOTICE. Not legal, tax or investment advice.