DRU-RE-EntityPair-TwoHead-AraModernBERT

DRU-RE EntityPair TwoHead

Arabic relation extraction model based on NAMAA-Space/AraModernBert-Base-V1.0.

Architecture

The marked sentence is encoded by AraModernBERT. The unchanged classifier explicitly extracts:

1. the contextual CLS vector,
2. the contextual opening subject-marker vector,
3. the contextual opening object-marker vector,
4. the absolute subject/object vector difference,
5. the elementwise subject/object vector product,
6. a 42-dimensional directional OneHot entity-type vector.

These are concatenated into 5H + 42 dimensions and compressed through a 1024 -> 512 pair MLP.

The shared pair representation feeds two heads:

• binary relation-existence head: no relation vs relation exists;
• 40-way positive-relation head: relation type, with no no_relation class.

For no_relation rows, only the existence loss is used. The positive-relation head is trained only on positive rows.

Threshold

The existence threshold is tuned on labeled val.jsonl to maximize positive micro F1. The threshold used for the official prediction was 0.0100 from auto_tuned_on_labeled_val.

Main files

• model/pytorch_model.bin
• model/modeling_entity_pair_two_head.py
• configs/run_config.json (includes the exact base encoder and resized tokenizer size)
• configs/architecture_config.json
• artifacts/existence_threshold_sweep.csv
• artifacts/submission.zip
• artifacts/predictions.txt