qwen3-5-0.8b-multitask-latex-chartqa

qwen3-5-0.8b-multitask-latex-chartqa

Qwen3.5-0.8B fine-tuned with LoRA (rank 32) on two tasks simultaneously: LaTeX OCR (image → LaTeX source) and ChartQA (image + question → short answer).

Results

Multi-task vs single-task cost (LaTeX EM): -0.5 pts (95% paired-bootstrap CI comfortably includes 0 — not distinguishable from no cost at n=200 eval samples)

⚠️ Important caveat on the "0% baseline" numbers

The baseline exact-match scores above look dramatic (0.0% on both tasks) but this is largely a formatting artifact, not zero underlying capability. The un-finetuned base model answers conversationally (e.g. "Based on the chart... ✅ Answer: 63") rather than with a bare terse string, so strict exact-match fails even when the content is correct. Under a lenient/normalized match, baseline ChartQA accuracy is closer to ~78%, not 0%.

What fine-tuning appears to primarily provide is output-format discipline (terse, direct answers) rather than new task knowledge — a LoRA adapter trained only on LaTeX OCR (never seeing a single ChartQA example) scores ~51% exact-match on ChartQA, almost entirely by unlocking latent chart-reading ability the base model already had. See assets/transfer_matrix.png and assets/examples.md for supporting qualitative examples.

Eval set size note

Evaluation used 200 held-out examples per task. Bootstrap 95% CIs on exact-match are ~13-14 points wide at this sample size, so small deltas (LoRA rank sweep, single- vs multi-task cost, seed variance) should be read as noise-level, not as confirmed effects.

Training config

• Base model: unsloth/Qwen3.5-0.8B
• LoRA rank: 32, alpha: 32
• Learning rate: 2e-4, batch size: 20, epochs: 1
• Train samples: 20000 LaTeX + 18000 ChartQA
• Training time: 23.6 min | Peak GPU: 21.686 GB (NVIDIA A100-SXM4-80GB)

See assets/ for loss curves, comparison plots, and qualitative best/worst prediction examples.