cua-s1-4b-coreml

cua-s1-4b-coreml

Core ML conversion of Cua-S1-4B-0.2 (Cua, Apache-2.0): LoRA adapters on Qwen/Qwen3.5-4B (Apache-2.0) that pick one (element, action) option for a GUI screen state. Both adapters are converted, each merged into its own copy of the base: text/ (accessibility tree) and multimodal/ (screenshot).

The model is a one-pass chooser, not a generator: one forward pass over the prompt, then a softmax over the answer-letter logits A..Z at the last position (Cua's FourBModel readout). The Core ML graphs keep exactly that: prefill only, no KV cache, no vocabulary head.

Runs through FluidUse (CuaS1FourBManager) on macOS 14+.

let cua = try await CuaS1FourBManager.load(configuration: .init(modality: .text, variant: "gptq"))
let decision = try await cua.decide(CuaS1FourBState(
    app: "portal", taskFamily: "login_auth", goal: "Log in as the demo user",
    accessibilityTree: "- [el_0] Edit \"Username\" value=\"\"\n- [el_1] Button \"Log in\"",
    options: [
        .init(elementId: "el_0", role: "Edit", label: "Username", action: "fill", entityId: "ent_0"),
        .init(elementId: "el_0", role: "Edit", label: "Username", action: "skip"),
        .init(elementId: "el_1", role: "Button", label: "Log in", action: "click"),
    ]))
print(decision.best?.option, decision.bestPerElement())

Files

Each decoder part takes hidden [1, L, 2560], cos/sin [L, 64] (interleaved M-RoPE, host computed) and returns the next hidden state; the last part also takes last_onehot [1, L] and returns letter_logits [1, 26]. Prompts are right-padded; the model is causal, so padding is exact.

Results

Apple M5 Pro (24 GB), macOS 27, GPU (cpuAndGPU). Parity reference: fp32 transformers decoder layers on the merged weights and the unpadded prompt, over 38 tasks from Cua's own cua_bench_s1 generator. Accuracy: GUI-360 test split (613 text tasks) rebuilt with Cua's gui360 converter; Cua's own frozen 615-task split is published by hash only, so this is protocol-adjacent, not the same task set.

One decision on the GPU: text ~0.7 s (1024-token bucket) and multimodal ~1.65 s (vision 0.2 s + 2048-token decoder) on an idle machine; about 1.1 s and 3 s while other GPU work was running.

On the same first 100 GUI-360 tasks, Cua's PyTorch runtime (bf16) scores 84%, Core ML fp16 88% and gptq 87%; the Core ML task outcome matches PyTorch on 96-97 of 100. Post-training int4 without calibration and 4/3/2-bit palettes are not published: they flip 10-32 of the 38 decisions.

Limits

• GPU only: the Neural Engine path falls back to the CPU for most of the graph (19 s / decision).
• At most 26 options per decision (one letter each). Longer prompts than the loaded bucket are rejected.
• Screenshots are smart-resized as in the reference processor, capped at 4096 patches (about 1280x800 px); larger screenshots are downscaled further than the PyTorch reference would.
• fp16 vision features move multimodal probabilities by up to 0.15 (argmax unchanged on the fixtures).

Provenance

Conversion code, parity and benchmark scripts: mobius models/computer-use/cua-s1-4b/coreml. Base weights: Qwen/Qwen3.5-4B; adapters: cua-ai/cua-s1-4b-0.2 (text/, multimodal/). Both Apache-2.0; see NOTICE.