ffdetr-filled

FFDetr — filled-form fine-tune

Fine-tune of jbarrow/FFDetr (RF-DETR Medium) that detects form fields even when they already contain values — typed text, handwriting, checkbox marks, signatures — on both born-digital and scanned pages. The base model only detects blank fields reliably; on filled fields its recall collapses to near zero.

Classes

Same head layout and index order as the base model:

Results

Evaluated on a held-out 300-image synthetic validation set (including 60 field-less negative pages) built from the CommonForms valid split — document-disjoint from training, with disjoint fill-content generator profiles. IoU 0.5, confidence 0.5:

• Overall [email protected]: 0.635 (base) → 0.815
• False positives on field-less pages: 1.47 → 1.32 per negative page
• On pages that do contain fields, absolute FP rises 438 → 502 (+15%) because the model now makes far more detections there — true positives rise 2669 → 3833 (+44%) — so the share of false boxes among all detections falls from 16.5% to 13.2%, and precision improves on every slice except blank_flat (flat: 0.90 → 0.89). The base model's low absolute FP on filled pages is an artifact of it barely detecting anything there.
• Inference latency unchanged (~57 ms/img on A100 at resolution 1024; same architecture, weights-only checkpoint)

On real filled documents (scanned bank forms with handwriting, typewriter-filled government forms), the fine-tune matches the base model on strongly-outlined blank fields and additionally detects value-bearing and graphics-obstructed fields the base model misses, without producing detections on field-less pages.

Training data

8,000 images derived from 4,000 pages of jbarrow/CommonForms (3,200 pages with fields + 800 negatives, revision-pinned). Every page is paired with a synthetic augmented counterpart: values rendered into field boxes (multilingual text incl. CJK/Arabic/Cyrillic, choice marks, signature strokes; partial-fill variants) and/or scan degradation (rotation, blur, contrast shift, noise, JPEG artifacts).

Recipe

Initialized from the base FFDetr checkpoint; detection head reinitialized for 3 classes. 8 epochs, lr 1e-5, batch 4 with grad-accum 4 (effective 16), resolution 1024, on 1× A100 40GB.

Usage

Pin a revision in production so a re-upload can never silently change your model:

from huggingface_hub import hf_hub_download
from commonforms.inference import FFDetrDetector

path = hf_hub_download(
    "pdf-net/ffdetr-filled",
    "FFDetr.pth",
    revision="e4fc1a8654915fe2f9bae63327062bc15711ecd6",
)
detector = FFDetrDetector(path)

The checkpoint is a drop-in replacement for the base model: same class indices, same input resolution, same inference code path.

Known limitations

• Detected boxes follow the CommonForms field-rect convention (the fillable area). Ink written outside that area — on or below an underline, spilling past the box — may not be fully covered by the box. A future revision trains with overflowing fills and ink-union target boxes.
• blank_scan recall is ~0.11 below the base model at conf 0.5 (precision is higher); expected to recover with the larger final training run.

Status

Interim checkpoint from an 8k-image pilot. A final fine-tune on a larger dataset is planned and will be published here as a new revision — pin a specific commit revision in production.

License: Apache-2.0, same as the base model (jbarrow/FFDetr), the CommonForms dataset, and the rfdetr training library.