transformers-ud-japanese-electra-base-ginza

transformers-ud-japanese-electra-ginza (sudachitra-wordpiece, mC4 Japanese)

This is an ELECTRA model pretrained on approximately 200M Japanese sentences extracted from the mC4 and finetuned by spaCy v3 on UD_Japanese_BCCWJ r2.8.

The base pretrain model is megagonlabs/transformers-ud-japanese-electra-base-discrimininator, which requires SudachiTra for tokenization.

The entire spaCy v3 model is distributed as a python package named ja_ginza_electra from PyPI along with GiNZA v5 which provides some custom pipeline components to recognize the Japanese bunsetu-phrase structures. Try running it as follows:

$ pip install ja-ginza-electra
$ ginza

Licenses

The models are distributed under the terms of the MIT License.

Acknowledgments

This model is permitted to be published under the MIT License under a joint research agreement between NINJAL (National Institute for Japanese Language and Linguistics) and Megagon Labs Tokyo.

Citation

@article{matsuda2020,
  title={GiNZA - Universal Dependencies による実用的日本語解析},
  author={松田 寛},
  journal={自然言語処理},
  volume={27},
  number={3},
  pages={695-701},
  year={2020},
  doi={10.5715/jnlp.27.695}
}

References

• UD_Japanese-BCCWJ

@inproceedings{asahara2018udjapanese,
    title = "{U}niversal {D}ependencies Version 2 for {J}apanese",
    author = "Asahara, Masayuki  and
      Kanayama, Hiroshi  and
      Tanaka, Takaaki  and
      Miyao, Yusuke  and
      Uematsu, Sumire  and
      Mori, Shinsuke  and
      Matsumoto, Yuji  and
      Omura, Mai  and
      Murawaki, Yugo",
    booktitle = "Proceedings of the Eleventh International Conference on Language Resources and Evaluation ({LREC} 2018)",
    month = may,
    year = "2018",
    address = "Miyazaki, Japan",
    publisher = "European Language Resources Association (ELRA)",
    url = "https://aclanthology.org/L18-1287/"
}

• mC4

Contains information from mC4 which is made available under the ODC Attribution License.

@article{2019t5,
    author = {Colin Raffel and Noam Shazeer and Adam Roberts and Katherine Lee and Sharan Narang and Michael Matena and Yanqi Zhou and Wei Li and Peter J. Liu},
    title = {Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer},
    journal = {arXiv e-prints},
    year = {2019},
    archivePrefix = {arXiv},
    eprint = {1910.10683},
}

• SudachiTra

@INPROCEEDINGS{katsuta2022chitra,
    author    = {勝田哲弘, 林政義, 山村崇, Tolmachev Arseny, 高岡一馬, 内田佳孝, 浅原正幸},
    title     = {単語正規化による表記ゆれに頑健な BERT モデルの構築},
    booktitle = "言語処理学会第28回年次大会(NLP2022)",
    year      = "2022",
    pages     = "",
    publisher = "言語処理学会",
}