DepNeCT-XLMR

XLM-R Adapter-Based Multitask Model (customized-mwt-ner)

This repository contains a fine-tuned XLM-RoBERTa model enhanced with task-specific adapter layers for multilingual NLP tasks including:

• Part-of-Speech tagging (UPOS, XPOS)
• Morphological features tagging (FEATS)
• Dependency parsing (DEPREL, HEAD)
• Named Entity Recognition (NER)

Trained using adapter-based finetuning and multi-task supervision on Universal Dependencies and custom NER data.

---

Files

• customized-mwt-ner.tagger.mdl
  → PyTorch state dict containing task adapter weights and classification heads.

• customized-mwt-ner.vocabs.json
  → Token and label vocabulary mappings (used for preprocessing and decoding).

---

Model Architecture

• Base: xlm-roberta-base (frozen)
• Adapter configuration: Pfeiffer
• Injected adapter layer per Transformer block
• Task-specific heads for:
  • POS (upos, xpos)
  • Morphological features (feats)
  • Dependency parsing (unlabeled, deprel)
  • Optional NER layer (if enabled)

---

Usage (PyTorch)

You’ll need to:

1. Load xlm-roberta-base from Hugging Face
2. Load adapter weights from customized-mwt-ner.tagger.mdl
3. Use the vocab file for decoding predictions

NOTE: This model is not directly plug-and-play with transformers unless you recreate the original architecture and adapter insertion logic.

Refer to the original paper for complete documentation.

---

Variants

This model comes in four variants:

---

License

APACHE 2.0 License. Please cite appropriately if used in research.

---

Citation

@misc{sandhan2023depnecti,
      title={DepNeCTI: Dependency-based Nested Compound Type Identification for Sanskrit}, 
      author={Jivnesh Sandhan and Yaswanth Narsupalli and Sreevatsa Muppirala and Sriram Krishnan and Pavankumar Satuluri and Amba Kulkarni and Pawan Goyal},
      year={2023},
      eprint={2310.09501},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Link to original paper => HERE
Link to DepNeCT Github Repo => HERE