homograph-bel-xlm-roberta-base

HomographBel XLM-R base

This model resolves the stress variant of one exact Belarusian homograph occurrence from its sentence context. It is intended as a component before phonemization in a Belarusian text-to-speech pipeline.

The input contract is text + exact target span + dictionary candidates. The output is a dictionary-approved stressed form, candidate probabilities, and an explicit support status. The model does not rewrite the complete sentence and does not predict stress for arbitrary out-of-dictionary words.

Model details

• Backbone: FacebookAI/xlm-roberta-base
• Architecture: contextual states at <target> and </target> markers, followed by a dictionary-masked variant classifier
• Orthography: Belarusian official orthography (official_2008)
• Dataset version: a747519e553a3532075e0f30517175ee93cdc93ce6ed448ab1ac7e912521b412
• Dictionary: GrammarDB-RELEASE-202601
• Training: 3 epochs, 4,785 optimizer steps, effective batch size 64, BF16 on an NVIDIA L40S
• Selection metric: gold-validation macro-F1 0.901211

The gold validation set contains only 275 contexts for 28 homographs. The reported metric is encouraging for those homographs but is not a broad estimate across all 1,941 trained homographs. Thirteen validation homographs expose only one observed variant. A sealed gold test score and a production confidence threshold are not published in this checkpoint.

Install

python -m pip install "huggingface_hub>=0.34,<2" "safetensors>=0.5,<1" \
  "torch>=2.7,<3" "transformers>=4.50,<5"

The repository is private during preliminary review, so authenticate first:

huggingface-cli login

Use

from huggingface_hub import snapshot_download

model_dir = snapshot_download("fosters/homograph-bel-xlm-roberta-base")

# inference.py is included in the downloaded model repository.
import sys
sys.path.insert(0, model_dir)
from inference import HomographResolver

resolver = HomographResolver.from_pretrained(model_dir, device="cpu")

text = "Найталенавіцейшы музыка і паэт."
start = text.index("музыка")
result = resolver.predict(text, start, start + len("музыка"))

print(result["stressed_form"])  # музы́ка
print(result["confidence"])
print(result["status"])         # ok

On Apple Silicon, device="mps" can be used when the installed PyTorch build supports MPS. Use device="cuda" on a CUDA host.

More examples

examples = [
    ("Найталенавіцейшы музыка і паэт.", "музыка"),
    ("Нават калі музыка ідзе фанаграмай.", "музыка"),
    ("Але гэта вучыць цябе дысцыпліне, адказнасці.", "вучыць"),
]

for text, target in examples:
    start = text.index(target)
    prediction = resolver.predict(text, start, start + len(target))
    print(target, "->", prediction["stressed_form"], prediction["confidence"])

Expected stressed forms for these contexts are музы́ка, му́зыка, and ву́чыць.

If the same target occurs more than once, pass the offsets of the occurrence that should be resolved. Python spans are half-open: text[target_start:target_end] must equal the target.

Training data

The corresponding private dataset is fosters/homograph-bel-contextual-v1. It contains the exact prepared train, silver-validation, gold-validation, and gold-test splits, the canonical dictionary, preparation manifest, and quality report.

Limitations and safety

• Most training labels are generated by Gemini models rather than human reviewers.
• FineWeb2 contexts are unfiltered web data and can contain toxic, sensitive, or identifying text. The training data should be audited before public release.
• The model is only supported for homographs and variants represented in accepted training evidence. Check status; do not treat unsupported_homograph or unsupported_variant as an automatic decision.
• Confidence is an uncalibrated softmax score. No production auto-accept threshold has been established.
• Candidate coverage comes from the bundled dictionary. The model must not invent variants outside it.
• The dataset combines sources with different licensing and attribution requirements. The repositories therefore use license: other pending a final legal/provenance review.

Reproducibility

The repository includes metadata.json, finetuning-v1.toml, the tokenizer, encoder config, runtime dictionary, support index, and safetensors checkpoint. The checkpoint SHA-256 is a72c3a5155a3cd5f926b4f34f9d743966ace4c466b7d76eb20b50e437c7d645c.

Citation

No formal citation is available yet. If you use this preliminary release, cite the model and dataset repository revisions together with the dataset ID above.