docred-stage3-atlop-fine-tuning-v5

Evaluation Results (Stage 3 — GAIN-lite, v5)

Evaluation was conducted on the DocRED dev set.

Dataset Statistics

• Documents: 998
• Predicted Triples: 10,671
• Gold Triples: 12,275
• Correct Predictions: 6,929

---

Main Metrics

• Micro F1: 60.39%

  • Precision: 64.93%
  • Recall: 56.45%

• Ign F1: 58.17%

  • Precision: 63.42%
  • Recall: 53.72%

• Evidence F1: 42.45%

  • Precision: 40.91%
  • Recall: 44.12%

---

Detailed Analysis

• Intra-sentence F1: 67.29%

  • Precision: 72.02%
  • Recall: 63.14%

• Inter-sentence F1: 52.06%

  • Precision: 56.28%
  • Recall: 48.42%

---

Interpretation

The Stage 3 (GAIN-lite) model continues to demonstrate stable performance on document-level relation extraction.

• The Micro F1 (60.39%) indicates consistent overall performance.
• The Ign F1 (58.17%) suggests reasonable generalization beyond memorized relations.
• The Evidence F1 (42.45%) shows that evidence prediction is functional but still remains a challenging component.

The gap between:

• Intra F1 (67.29%)
• Inter F1 (52.06%)

indicates that:

• The model is strong at within-sentence relation extraction
• But cross-sentence (multi-hop) reasoning still has room for improvement

---

Key Insight

Stage 3 introduces graph-based reasoning, which improves entity interaction modeling.
However, the remaining gap in inter-sentence performance highlights the limitation of shallow GNN layers in capturing long-range dependencies.

---

Output Files

• Predictions: results/stage3/v5/predictions.json
• Submission File: results/stage3/v5/result.json