one-pass-sv-forms

One-Pass SV-Forms (SV0): a Swedish one-pass form specialist (research checkpoint)

This is a research checkpoint, not a product. It is a 706 048-parameter model that scores a supplied list of options in one forward pass, using a System One-style interface. It was trained from scratch on synthetic Swedish forms. It generates no text, needs no tokenizer, and answers in a measured median of 1.314 ms per prediction on an Apple M4 with CPU_AND_NE (Core ML packages: about 1,472 KiB fp16, 787 KiB int8). This excludes input encoding, model loading and warm-up; it is not an end-to-end form completion time.

Write-up

A Swedish form specialist in one forward pass: the engineering write-up behind this checkpoint, with the figures and the same pinned sources.

Measured results (our harness, Apple M4)

The shuffled-context control rotates contexts between rows while retaining their options and labels. Accuracy falls to 31.99%, showing that the model uses the context rather than only option statistics. It does not identify which context clues the model has learned. The synthetic test contains 13,844 expected fill decisions; one is answered skip. This count excludes choosing a wrong fill value. Fill accuracy is 99.00%.

For comparison, the unchanged released English checkpoint scores 20.75% on the same 29,839 rows, with 5,412 of 13,844 expected fills skipped. Labels, action strings and the generated distribution all change; the result does not isolate language as the cause. The 50 handwritten decisions comprise 32 fills, 12 skips, three checks and three clicks. They are a small diagnostic set, not a broad test of Swedish forms.

Data-size curve (same 4-epoch schedule, same seed, corrected generator)

This checkpoint is the 10,000-episode point. Each run has its own generated test set. At a fixed four epochs, the larger training sets also receive more optimization steps. These runs do not isolate the effect of data size or establish convergence.

Changelog

• v2 (2026-09-21): corrected synthetic identifier formats and increased the corpus to 10,000 episodes. The generator previously emitted malformed personnummer- and organisationsnummer-shaped values. Tests now check the intended formats and checksums. Such formatting differences could provide shortcuts, but the runs do not measure how much the model used them. Comparing the earlier 83.02% run with the corrected 900-episode run at 77.10% does not establish a cause for the difference: the generated training and test examples changed. Larger subsequent runs also used more training steps.
• Core ML clarification: int4 was initially evaluated with CPU_AND_NE and described as losing too much weight precision. The same package reaches 99.27% on CPU, versus 49.92% with CPU and Neural Engine allowed. Actual operation placement and the root cause remain unknown. Int8 and fp16 are not prediction-identical. See the public execution investigation for the full M4 comparison, M1 Max follow-up, decompression control and remaining tests.
• v1 (2026-09-21): earlier 900-episode release, 83.02% top-1 and a reported 7.5% silent-skip rate. Its artifacts remain in repository history. Use each revision's own result files and denominators when comparing releases.

What it is NOT

• Not validated for real form completion. Training data are generated, not collected from people or customer submissions. Names are constructed and email addresses use .invalid. Generated identifiers are not checked against registries; coincidental matches with real identifiers cannot be ruled out. The external checks are small: 50 handwritten decisions and 270 decisions in simplified fixtures based on Kanslist and Pratsam forms. Their labels and controls differ from the current sites. They use synthetic scenarios and hand-mapped expected actions, not real submissions or browser runs. The fixture results are 52.8% and 77.3%, with 57 of 65 expected Kanslist fills answered skip. These are expected fills, not necessarily HTML-required fields. Pratsam's fixture also has a gold-label defect: its scenario uses mig själv, while the radio mapping checks for ja, labelling all 15 radio rows skip. Those results are provisional. Vocabulary, input lengths, roles and fixture conventions need investigation; the current evidence does not isolate one cause.
• Not an agent. It does not generate text, cannot invent a value it was not given, and does not decide execution order: the option list and the sequence come from ordinary code around it.
• Not safe unsupervised. One expected fill out of 13,844 was answered "skip" in the synthetic test: a silent failure. Any real integration must check outcomes outside the model (fail closed, dry run, one submit, human review before consequential actions).
• Not tuned for throughput. One decision per predict call is what the latency numbers describe; batching is untested.

Core ML export (same checkpoint, coremltools 9.0, Apple M4)

All 29,839 held-out decisions are included, with the same encoding and masked argmax. CPU_AND_NE allows CPU and Neural Engine use; it does not report which operations ran on which processor. Package sizes sum the files, not runtime memory. One KiB is 1,024 bytes.

The int8 recipe uses a uniform 256-entry palette per tensor with float16 computation. Int4 uses grouped-channel K-means palettes, groups of 32, and a different deployment target: iOS 18, versus iOS 17 for fp16 and int8. These are different export recipes, not only different bit widths.

The same int4 package preserves 99.27% top-1 on CPU but falls to 49.92% when CPU and Neural Engine are allowed. That is an execution-dependent discrepancy, not evidence that these rounded weights cannot represent the task. Int8's rounded top-1 matches fp16 under CPU_AND_NE, but it changes 13 choices versus PyTorch; it is not lossless or prediction-identical to fp16.

A follow-up on M1 Max, macOS 26.5.1, used 512 deterministic samples spread across the test set. Int4 scored 511/512 on CPU and 266/512 with CPU_AND_NE. Decompressing its palettes to dense constants retained exactly those choices under each setting. This removes the palette-expansion operations but keeps the rounded weights and iOS 18 target. It also runs conversion passes, so it is not proof of an otherwise identical graph. ANECCompile() FAILED messages were observed, but the buffered output could not be attributed to individual calls. Neither these messages nor the predictions establish operation placement, fallback or a faulty operation.

See the public investigation for pinned data, scripts, artifact hashes and the remaining target-matched export and graph-inspection work. The chip and OS both differ between the M4 and M1 Max tests; their effects are not isolated.

The original M4 latency run excluded three warm-up rows, leaving 29,836 predictions. Int8 with CPU_AND_NE had median 1.314 ms and p95 1.411 ms; fp16 on CPU had median 1.727 ms. These time MLModel.predict, not preprocessing, model loading or a full form.

Usage

from pathlib import Path
from huggingface_hub import hf_hub_download

weights = Path(hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.safetensors"))
hf_hub_download("precisit/one-pass-sv-forms", "sv0-forms.json", local_dir=weights.parent)
# then either use the Core ML packages (no Python runtime needed, see example.py) or the PyTorch
# code and the full recipe in the toolkit repository:
# https://github.com/precisit/one-pass-specialists

The context string and options must be built exactly as in training (byte ids = UTF-8 byte + 1, zero padded; UPPGIFT fyll i formuläret från dokumentet och skicka sedan in / FORM <titel> / ELEMENT <roll> "<etikett>" value="…"). An encoding mismatch is one possible source of poor output; vocabulary, truncation, fixture labels and runtime behavior also need checking. The generator, catalogue and scoring harness are in the toolkit repository.

Building your own

precisit/one-pass-specialists is the public toolkit that produced this checkpoint: catalogue, synthetic generator with form-disjoint splits, trainer, evaluation (including the shuffled-context control and the silent-skip count), Core ML export with a 512-example check of the export-friendly eager implementation against its trace, followed by Core ML decision comparisons, and the SV0 recipe. The trace check does not establish bit identity with the original unmodified forward pass. The synthetic corpus itself is published as precisit/one-pass-sv-forms-synthetic.

Licence and attribution

MIT. The architecture, training loop and evaluation metrics are vendored unmodified from Cua's MIT-licensed libs/cua-s1 (which credits jevlike for the option-attention head design). The Swedish catalogue, synthetic generator, training runs, measurements and Core ML tooling are Precisit's. No TypeSafe AI code, weights, data or API output was used: "Jev" and "System One" are their names for a similar interface, and this is an independent implementation. See THIRD_PARTY_NOTICES.md.