Jev-Style-Qwen3.5-2B-Decision-v2-MLX-bf16

Jev-Style-Qwen3.5-2B-Decision v2 (MLX BF16)

<!-- jevstyle:project-links:start -->

Local classification, routing and fixed-choice scoring with JevStyle.

This is the archived v2 2B release (2026-09-23), maintained by Chaoliang Yan; it used no Jev outputs. It is kept for existing integrations and runs only through the clients shipped in this repository: the jev-style package does not load it (it loads the v3 and v4 releases). The current recommended release is 2B v4 (text and images; trained in part on soft labels distilled from TypeSafe Jev 1.14 through its public API) and, for the smallest text model, 0.8B v3. JevStyle is independently maintained by Chaoliang Yan, with no TypeSafe affiliation or endorsement. Training provenance is release-specific.

Official local installation of the current releases — no hosted API key or agent simulation: python -m pip install -U "jev-style[torch]" and jev-style serve --release 2b (2B v4; --release 0.8b for the 0.8B v3), or python -m pip install -U "jev-style[mlx]" on Apple silicon for text. The package runs downloaded JevStyle weights on your machine. Step-by-step installation and verification. TypeSafe/OpenRouter Jev access and skills that simulate decisions with another agent model are separate execution paths.

Hosted service is separate from local inference. Hosted API: operational. It serves the 2B v4 (text only; 8,192-token context, inputs up to 8,000 tokens). There is currently no fixed per-account quota: no daily request count and no per-minute cap. Capacity is shared, so a busy server answers 429 with a Retry-After header and clients should retry. Check current service status or live health when using the hosted API.

Choose a workflow: Python text classification, local routing tools and Apple silicon, receipt or screenshot yes/no decisions, reproducible routing comparison. These guides distinguish suitable alternatives and published evidence; they do not establish universal superiority.

<!-- jevstyle:project-links:end -->

Website: jevstyle.com/models — all JevStyle decision models, benchmarks and quickstart in one place. This v2 repository runs through the clients it ships (below); the jev-style package does not load it.

A Jev-style decision model for classification, routing and typed choices. Give it a state, a question and a list of options; one prefill returns a selected option with calibrated probabilities.

Download this build: model.safetensors. The repository also includes its calibration, inference client and evaluation records.

Results

81.20% macro accuracy on the fixed English reference panel, compared with 76.68% for v1 and 75.09% for English Laya. The results below use the CUDA reference structure: 11 real-label task groups, 3,277 decisions, equal task weights, and the same 3,100-record calibration split. Results for the released deployment formats appear further below.

• Higher accuracy: +4.53 percentage points over v1 and +6.12 over English Laya; paired 95% intervals are [+3.58, +5.52] and [+4.64, +7.52] points, respectively, within this fixed panel.
• Broader task coverage: accuracy point estimates ahead of English Laya in 9 of 12 task groups, including the separately scored teacher-reference typed-decisions group.
• Better probability quality against English Laya: 18.4% lower NLL, 20.0% lower Brier score, and 26.4% lower task-macro ECE.
• Efficient adaptation: 36.9 minutes of main training on one H100 80GB, using rank-32 LoRA on a 2B-class text backbone.

Calibration

The reliability diagram plots the v2 model's stated confidence against observed correctness. Every real-label evaluation decision is included; the histogram shows how many predictions fall in each confidence bin. Error bars show Wilson 95% intervals. The accompanying ECE comparison averages per-task calibration errors.

Temperature is fitted on the calibration split. HF and MLX clients apply the supplied calibration automatically; the calibrated GGUF file incorporates it in the final normalization tensor.

Robustness

Option-order flip rate is halved relative to English Laya, with 80.00% accuracy after permutation on the same 400 Choice/Bool decisions. Semantic options are mapped back to their original identities before scoring.

On a separate 200-pair programmatic threshold-policy test, both decisions in a counterfactual pair are correct in 71.50% of pairs for v2, compared with 63.00% for v1. This test measures that specific rule family.

Task-level results

<details> <summary><strong>Per-task accuracy: all 11 real-label tasks and the separate typed-decision group</strong></summary>

The separate typed-decisions group contains 2,000 teacher-reference decisions from 400 states. Teacher agreement is 53.35% for v1, 37.55% for English Laya and 73.45% for v2 under the fixed primary interface. This group is excluded from the real-label macro. The comparison here uses the English Laya checkpoint; specialist-checkpoint and rendering sensitivity results are provided in baseline_sensitivity.json.

</details>

Deployment validation

GGUF is available in Q4_K_M, Q8_0 and BF16, each with its own calibration and verification record.

Each deployment format has its own validation record. Native MLX packaging reproduces the verified MLX client's logits exactly on all 500 deployment cases. The Q8_0 model is approximately 46.7% smaller than the BF16 GGUF export.

On the same 500-case deployment subset, real-label task-macro accuracy is 79.10% for CUDA BF16, 79.04% for MLX BF16 and 78.69% for Q8_0. Full-panel reference results and deployment-subset results use their respective denominators.

<details> <summary><strong>Evaluation data and downloadable vector charts</strong></summary>

• Reference metrics and paired intervals
• Deployment validation
• Baseline sensitivity results
• Data sources and split manifest
• Chart data, confidence bins and sample counts
• Vector charts: benchmark, calibration, robustness

The benchmark figures describe the fixed CUDA reference comparison. Reliability pools all real-label examples into confidence bins; task-macro ECE is the mean of 11 separate task ECE values. These are distinct aggregations. The 9/12 figure counts task-level point estimates. Individual prediction probabilities, task summaries, test protocols and calibration records were retained when drawing these charts.

</details>

Quick start

python -m pip install -U huggingface_hub
hf download chaoliangUNSW/Jev-Style-Qwen3.5-2B-Decision-v2-MLX-bf16 --local-dir jev-v2-mlx
cd jev-v2-mlx
python -m pip install -r requirements.txt
python jev_mlx_client.py --model . \
  --state "The film was excellent." \
  --question "What is the sentiment of this review?" \
  --options negative positive

from jev_mlx_client import Client

model = Client(".")
result = model.decide(
    "The film was excellent.",
    "What is the sentiment of this review?",
    ["negative", "positive"],
)
print(result["choice"])
print(result["probabilities"])

Validated with mlx==0.32.2 and mlx-lm==0.31.3. The companion calibration.json is loaded automatically. The reference benchmark and MLX deployment check are reported separately above and in the evaluation files.

Decision interface

Provide an English state, a question, and 2–26 unique options, within a 1,024-token prompt. A single prefill produces one logit per declared option. Apply the supplied calibration once and normalize over those options to obtain the decision probabilities.

You are a decision function. Read the state, then answer the question by choosing exactly one option.

[State]
The film was excellent.

[Question]
What is the sentiment of this review?

[Options]
A. negative
B. positive

Answer:

The supplied clients implement this exact prompt and read the next-position A through Z token scores. Use this decision interface for Choice, Bool and ordered Score tasks; decide_bool returns the probability of yes, and decide_score also returns the expected zero-based level. For ordered scores, supply options from lowest to highest.

Training

Continued from the uncalibrated Jev-Style v1 text backbone derived from Qwen3.5-2B-Base. Training used a BF16 backbone, FP32 rank-32 LoRA (alpha 32), 186 adapted modules and 33,638,400 trainable parameters. Effective batch size was 64 with a 1,024-token budget.

The 60,000-record training pool combines original-task replay with emotion, email, typed workflow decisions, label transformations and programmatic threshold rules. A two-stage schedule increases hard-example sampling while retaining approximately 50% original-task replay. Main training completed 1,000 optimizer updates and processed 11,605,632 tokens in 36.9 minutes on one H100 80GB.

Development (2,050 records), calibration (3,100 records) and final evaluation (5,277 decisions) were handled separately. Checkpoint selection used development results; calibration used the calibration split. This release records one training seed. The evaluation JSON files document the dataset, rendering and deployment protocols.

License and attribution

Apache-2.0. See LICENSE. This release builds on Qwen3.5-2B-Base and Jev-Style v1. Training data retain their original source licenses; source and split details are recorded in the data manifest. The release contains model artifacts and aggregate evaluation records.

Contact

I welcome internship, employment, and research collaboration opportunities. Please contact me at [email protected].

欢迎提供实习、工作及科研合作机会，请邮件联系：[email protected]。