unrender-qwen3-4b-lora-v2

unrender — Qwen3-4B LoRA v2

LoRA adapters that reconstruct compilable LaTeX from a rendered PDF's extracted text, word boxes, and declared font names.

• Base: mlx-community/Qwen3-4B-Instruct-2507-4bit (QLoRA over a 4-bit base)
• Adapters: 3.67M params (0.091%), last 8 layers, 4096 context, prompt-masked
• Training data: 91 examples from 254 repos; every target's \documentclass verified to resolve system-wide; no eval source appears in training
• Code: https://labs.gauntletai.com/cameronmoore/unrender
• Eval-code commit: ab44ce9284e69ac69d87f93865310088c3bf002b

Results — 34 held-out scenarios, zero training overlap

What it does

Near-eliminates the base model's silent glyph loss — 96 characters dropped across the eval set, down to 4. XeTeX drops a glyph the font lacks with only a log warning, never an error, so the document compiles and the character is simply gone. Font-related errors fall from 89% of all failures to about 9%.

That failure mode is the one prompting could not fix. In a 204-run ablation across two model families, a strategy that states the rules verbatim — load fonts by filename with Path=, route missing glyphs through substitution macros — scored worst of three strategies (0.3442 vs 0.4903 for few-shot). Being told the rule did not install it; training on 91 examples did.

What it does not do

It does not learn layout. right_align measures the resume date-column idiom — employer flush left, date flush right on the same baseline — and this model scores 0.156 where the reference is 1.0 and a frontier model reaches 0.696. It also regresses compile rate versus its own base (59% → 47%).

It is 7.6× below a cheap frontier model on the same eval set. This closes a gap to its own base, not to the frontier. Published as an honest checkpoint, not a recommendation to use it instead of prompting a larger model.

Usage

Requires Apple Silicon (MLX format; not loadable by peft/transformers without conversion).

pip install mlx-lm

from mlx_lm import load, generate
model, tok = load(
    "mlx-community/Qwen3-4B-Instruct-2507-4bit",
    adapter_path="path/to/this/repo",
)

Input is an extraction, not a raw PDF — page geometry, declared font names, and line boxes. See src/unrender/extract.py in the code repo.

Reproduce

./eval.py --model mlx-community/Qwen3-4B-Instruct-2507-4bit \
          --adapter-path adapters/qwen3-4b-v2-best \
          --eval-set data/eval/ablation.jsonl

Limitations

1. Far below frontier models; see the table above.
2. Layout reconstruction essentially does not work.
3. Compile rate regressed relative to the base model.
4. MLX-only; Apple Silicon required.
5. Trained on public LaTeX resume sources, English-dominant.