koelectra-small-v3-privacy-ner

KoELECTRA-small-v3-privacy-ner

This model is a fine-tuned version of monologg/koelectra-small-v3-discriminator on a synthesized privacy dataset. It achieves the following results on the evaluation set:

• f1 = 0.9998728608843798
• loss = 0.05310981854414328
• precision = 0.9999237126509853
• recall = 0.9998220142897098

Model description

태깅 시스템 : BIO 시스템

• -B(begin) : 개체명이 시작할 때
• -I(inside) : 토큰이 개체명 중간에 있을 때
• O(outside) : 토큰이 개체명이 아닐 경우

12가지 한국인 개인정보 패턴에 대한 태그셋

How to use

You can use this model with Transformers pipeline for NER.

from transformers import AutoTokenizer, AutoModelForTokenClassification
from transformers import pipeline

tokenizer = AutoTokenizer.from_pretrained("amoeba04/test1")
model = AutoModelForTokenClassification.from_pretrained("amoeba04/test1")
ner = pipeline("ner", model=model, tokenizer=tokenizer)

example = "지난주, 홍길동 씨는 서울특별시 강남구에 위치한 테헤란로 101빌딩에서 진행된 IT 컨퍼런스에 참석했습니다."
ner_results = ner(example)
print(ner_results)

출력: "PER-B, PER-B 씨는 LOC-BLOC-ILOC-I LOC-ILOC-I LOC-ILOC-I LOC-ILOC-I LOC-ILOC-ILOC-I에서 진행된 IT 컨퍼런스에 참석했습니다."

Training and evaluation data

자체 제작한 한국인 개인정보 패턴 기반 개체명 인식 (NER) 데이터셋

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 512
• eval_batch_size: 1024
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1
• mixed_precision_training: Native AMP

Framework versions

• Transformers 4.40.0
• Pytorch 2.2.1+cu118
• Datasets 2.19.0
• Tokenizers 0.19.1

Authors and Acknowledgments

This model was developed by the KNU Vision & Learning (KVL) Lab at Kyungpook National University.

For more information about our work, please visit our website.