convnextv2-polyline-base-hebrew-samaritan

convnextv2-polyline-base-hebrew-samaritan

Hebrew + Samaritan fine-tune of convnextv2-polyline — ConvNeXtV2-Base backbone with an FPN neck and a D-FINE-style polyline decoder, predicting text-line baselines as polylines (control points + height) for medieval Hebrew and Samaritan manuscripts (PAGE-XML).

Hub repo: johnlockejrr/convnextv2-polyline-base-hebrew-samaritan
Primary file: best_cbad_f1.safetensors (~372 MB, FP32)
SHA256: b092a49b1cf578d569283e166e3d84e3e33d2fede097955b060b2d8f56e49b5c

Init chain: timm ConvNeXtV2-Base (fcmae_ft_in22k_in1k) + partial decoder load from johnlockejrr/dfine-det-large-baseline-stage0 → this fine-tune.

The Stage-0 checkpoint is a D-FINE Large / HGNetv2 polyline model. Only decoder tensors that match shape transfer into this ConvNeXt + FPN stack (backbone and FPN train from ImageNet / scratch). This release is not a continuation of the dfine-det Stage-1 Hebrew product weights.

Not for the ICDAR cBAD / Orli bake-off table. Scores below are on the Hebrew+Samaritan combined validation split used for early stopping.

---

Model summary

Detection is set prediction of scored polylines. Polygonal line environments (BLLA) are applied only at PAGE/ALTO serialize time, not in the training objective.

---

Intended use

Use for

• Baseline detection on Hebrew and Samaritan manuscript pages (PAGE-XML workflows)
• Downstream HTR / layout pipelines that need text-line geometry without a D-FINE / HGNetv2 stack
• Research comparing ConvNeXtV2 + FPN vs HGNetv2 + HybridEncoder for the same polyline head and Arrow data

Not for

• Official ICDAR cBAD 2019 bake-offs (wrong track / domain)
• Production OCR transcription (detects lines; does not recognize text)
• Replacing layout region detectors (paragraphs, tables, etc.)
• Drop-in replacement of dfine-det Hebrew Stage-1 without re-validating conf and domain gap

---

Training pipeline

timm ConvNeXtV2-Base (ImageNet-22k → ImageNet-1k FCMAE FT)
        │
        ├─ backbone weights
        │
dfine-det Large Stage-0 polyline pretrain
  Hub: johnlockejrr/dfine-det-large-baseline-stage0
  (decoder tensors only → PolylineTransformer; FPN/backbone skipped)
        │
        ▼
Hebrew + Samaritan fine-tune  ← this release
  (1758 train / 261 val; monitor peak cbad_f1_max ≈ 0.8942 @ ep239, conf 0.50)

Config: configs/hebrew_samaritan_stage0_init_base.yaml in the convnextv2-polyline repository.

For reference, the same data + Stage-0 decoder init on ConvNeXtV2-Tiny peaked near cbad_f1_max ≈ 0.79. This Base run is the scaled backbone experiment.

---

Fine-tuning data

PAGE-XML baselines from:

Split policy (seed 42; see dfine-det configs/hebrew_samaritan_combined_split.json):

• Samaritan: manuscript-level holdout (~12% pages target) — 43 train MSS / 7 val MSS (no page from a val MS appears in train).
• Hebrew: page-level split within Hebrew_Medieval-seg.

Compiled to Arrow (simplify_eps=0.01 → (K=8) control points) under dfine-det outputs/hebrew_samaritan_combined/arrow/ (schema-compatible with convnextv2-polyline).

---

Training recipe

Training was resumed once from checkpoints/last.ckpt after an interrupt (TensorBoard version_0 → version_1); metrics above are from the completed run through epoch 248.

---

Evaluation

Metric: cBAD-style line F1 — Hungarian matching of densified polylines with mean bidirectional Chamfer cost; TP if cost ≤ dist_thresh (20 px).

Hebrew + Samaritan combined val (early-stop split)

261 pages (hebrew_samaritan_combined val Arrow). Post-hoc test --conf-sweep --fast-conf-sweep on best_cbad_f1.safetensors (EMA export).

Headline (val @ 0.50): F1 0.898 · P 0.937 · R 0.862

Locked operating point: conf = 0.50. Conf 0.40 is within ~0.001 F1 (more balanced P/R). Train monitor peak was cbad_f1_max ≈ 0.8942 @ ep239 (same conf); post-hoc EMA eval is slightly higher. Fixed log tag cbad_f1 (conf 0.1) is much lower — always sweep or use 0.50.

Caveats

• These figures are in-domain val (same split used for early stopping). They are optimistic vs a fully held-out test set.
• An unseen Hebrew multi-export holdout (as used for dfine-det Stage-1) was not scored for this release; re-run test on that Arrow before comparing to Stage-1 unseen F1 (~0.90).
• Val F1 here (~0.90) is not interchangeable with dfine-det Stage-1 val (~0.95) — different backbone/neck and different init.
• Final epoch (248) train-time cbad_f1_max was 0.8828 (slightly below the ep239 peak).

---

How to use

Install

# From the convnextv2-polyline repository (PyTorch CUDA 12.6 recommended; see scripts/fresh_venv_cu126.sh)
uv sync
# or: pip install -e .

Inference (PAGE XML)

convnextv2-polyline -c configs/hebrew_samaritan_stage0_init_base.yaml -d cuda:0 infer \
  -w best_cbad_f1.safetensors \
  -o page.xml \
  --format page \
  --conf 0.5 \
  --reading-order rtl \
  page.jpg

Evaluate

convnextv2-polyline -c configs/hebrew_samaritan_stage0_init_base.yaml -d cuda:0 test \
  -w best_cbad_f1.safetensors \
  -e /path/to/val.arrow \
  --conf 0.50

Conf sweep:

convnextv2-polyline -c configs/hebrew_samaritan_stage0_init_base.yaml -d cuda:0 test \
  -w best_cbad_f1.safetensors \
  -e /path/to/val.arrow \
  --conf-sweep --fast-conf-sweep

---

Files in this release

Upload the safetensors (+ this README) for Hub users; Lightning .ckpt files are optional (~1.1 GB each).

---

Limitations

• In-domain val F1 (~0.89) will overstate performance on unseen collections until a holdout is measured.
• (K=8) control points underfit strongly curved / damaged lines.
• Vertical “ghost” doubles can still appear on some layouts; cost_y / loss_height mitigate but do not eliminate them.
• FPN neck (vs D-FINE HybridEncoder) and partial Stage-0 transfer leave a gap vs the HGNetv2 Stage-1 product model on the same data.
• Private / institutional manuscript images used in training are not redistributed with the weights; respect each corpus license.
• Do not cite these scores as cBAD 2019 / Orli bake-off results.

---

Citation & credits

ConvNeXt V2 (backbone)

@inproceedings{woo2023convnextv2,
  title     = {ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders},
  author    = {Woo, Sanghyun and Debnath, Shoubhik and Hu, Ronghang and Chen, Xinlei
               and Liu, Zhuang and Kweon, In So and Xie, Saining},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2023}
}

• Weights via timm: convnextv2_base.fcmae_ft_in22k_in1k

D-FINE (polyline decoder lineage)

@inproceedings{peng2025dfine,
  title     = {D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinement},
  author    = {Peng, Yansong and Li, Hebei and Wu, Peixi and Zhang, Yueyi and Sun, Xiaoyan and Wu, Feng},
  booktitle = {The Thirteenth International Conference on Learning Representations},
  year      = {2025},
  url       = {https://arxiv.org/abs/2410.13842}
}

• Code: github.com/Peterande/D-FINE
• Weights: huggingface.co/Peterande/D-FINE

Stage-0 polyline pretrain (decoder init)

@software{dfine_det_stage0_large,
  title  = {dfine-det Large Stage-0: Multiscript Polyline Baseline Pretrain},
  year   = {2026},
  url    = {https://huggingface.co/johnlockejrr/dfine-det-large-baseline-stage0}
}

This ConvNeXtV2-Base Hebrew/Samaritan fine-tune

@software{convnextv2_polyline_hebsam_base,
  title  = {ConvNeXtV2-Base Polyline: Hebrew and Samaritan Baseline Detection},
  author = {johnlockejrr},
  year   = {2026},
  note   = {ConvNeXtV2-Base + FPN + D-FINE polyline decoder; Stage-0 decoder init; operating conf 0.50},
  url    = {https://huggingface.co/johnlockejrr/convnextv2-polyline-base-hebrew-samaritan}
}

Additional notices

• Polyline head, matcher, losses, Arrow I/O, and PAGE/ALTO export are vendored from dfine-det (Apache-2.0), itself deriving from D-FINE (Apache-2.0, © 2024 The D-FINE Authors).
• PAGE polygonalization helpers derive from Kraken/BLLA (Apache-2.0, © Benjamin Kiessling) — used at export only.
• Training corpora: follow each dataset’s original license / institutional terms.

---

License

Apache License 2.0 for the convnextv2-polyline code and these weights, consistent with D-FINE’s Apache-2.0 release and the Apache-2.0 ImageNet fine-tuned ConvNeXtV2 checkpoints redistributed via timm. Downstream users must also comply with licenses of any datasets used in further fine-tuning.