effect-qwen36-35b-mlx

Effect-v4 Qwen3.6-35B-A3B — Champion (MLX, 4-bit)

A local model fine-tuned to write idiomatic, compiling Effect v4 ([email protected]) TypeScript. Built $0-local on a single Apple M5 Max (48 GB): continued-pretraining → instruction SFT (LoRA), then fused into the 4-bit base in native MLX format. This is the exact artifact our serving stack runs.

Why this exists: [email protected] is a beta that postdates the pretraining of essentially every LLM — its exact API surface is absent from base models, so they hallucinate v3-isms. That sparsity is the whole point: this is a small, honest domain expert for a library the big models haven't seen.

Honest framing first. These are the fine-tuned weights. On a frozen, real-tsc --strict compile gate (24 held-out tasks) they are a genuine but limited expert: single-greedy + RAG ≈ 9.7/24 mean (best checkpoint 13/24). The headline ~23/24 number is the full serving pipeline (best-of-16 sampling + retrieval + a deterministic import-resolver + a tsc verifier), not the bare weights — see How to actually get 23/24. Treat this as a research artifact, strongest when paired with retrieval and a compiler-in-the-loop.

What it is

• Base: mlx-community/Qwen3.6-35B-A3B-4bit — the text tower of the Qwen3.6 hybrid GatedDeltaNet MoE (qwen3_5_moe, 35.9B total / ~3B active). Vision tower dropped (text code model).
• Fine-tune (champion v7s43_i200): CPT on a curated Effect-v4 source corpus (effect-smol, EffectPatterns, examples) → instruction SFT (rank-8 LoRA, 423 gate-validated instruction→code pairs, every target compiled under the exact tsc gate).
• This repo: the champion LoRA fused into the 4-bit base, in native MLX format (~18 GB). Runs directly with mlx_lm on Apple Silicon — no conversion, no dequantization.

Format note (MLX vs GGUF): the model was trained and is served in MLX, so this repo is the native, canonical release. A portable llama.cpp GGUF build is now also available (Q4_K_M + Q8_0, smoke-verified) at jrad123777/effect-qwen36-35b-gguf — use that for CPU / non-Apple inference.

Eval (real tsc --strict, frozen 24-task held-out benchmark)

Raw single-greedy + RAG — honest, same-harness, multi-seed flat mean (never a cherry-picked run). This is the bare-weights number; the ~23/24 headline is the serving pipeline below, not this table:

The dominant residual failure is decoding discipline (the model knows the API — best-of-N reaches 22–24/24 — but greedy decoding sometimes omits a namespace import). This is closed by tooling, not by more training: every $0 in-weights lever (more data, self-distillation, external real-repo data, RAG-tuning, decode-time constraints) was tested and plateaus here. Pushing raw ≥15 needs RL-with-compiler-reward (out of $0-local scope).

How to actually get 23/24

The production pipeline (open-source in the training repo, serve/serve.py) wraps these weights with:

1. best-of-16 sampling (temp 0.8 / top-p 0.95) — tsc is a perfect verifier; keep any sample that compiles,
2. targeted RAG over tsc-gated Effect-v4 idiom snippets,
3. a deterministic import-resolver (fixes TS2307/TS2304 namespace imports),
4. an optional 1-pass tsc-feedback repair.

That stack reaches ~23/24 on the broad served product. This repo gives you the expert weights; add your own best-of-N + a compiler check for production use.

Usage (mlx_lm, Apple Silicon)

pip install mlx-lm

# one-shot generation
python -c "
from mlx_lm import load, generate
model, tok = load('jrad123777/effect-qwen36-35b-mlx')
prompt = 'Write Effect v4 code: a Schema.Struct for a User with branded UserId, decoded with Schema.decodeUnknownEffect.'
msgs = [{'role':'user','content':prompt}]
text = tok.apply_chat_template(msgs, add_generation_prompt=True, enable_thinking=False)
print(generate(model, tok, text, max_tokens=512, verbose=True))
"

# or an OpenAI-compatible server
mlx_lm.server --model jrad123777/effect-qwen36-35b-mlx --port 8080

Tip: for best results, prepend a few real Effect-v4 example snippets (RAG) and sample N times keeping the first that compiles under tsc.

Limitations

• Single-greedy compile rate is ~⅓–½ of hard tasks; pair with RAG + best-of-N + a tsc gate.
• [email protected] only; later betas may shift APIs.
• Reasoning/thinking is disabled (enable_thinking=False) — it's a direct code generator.
• MLX format → Apple Silicon. For CPU / non-Apple, use the portable GGUF (see format note above).

Supersedes jrad123777/effect-qwen36-35b-v3-gguf (an earlier, weaker SFT checkpoint).

Built $0-local. Trained, evaluated against the installed .d.ts with tsc as the only arbiter, and documented honestly.