roberta-base-job-ner

Model Card for Model ID

<!-- Provide a quick summary of what the model is/does. -->

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This entity recognition model is used for information extraction in job-related texts. It can identify entities of Occupation, Skills, Qualifications, Experience and Domain. It is trained based on the open-source dataset provided by Green et. al.

• Developed by: International Hellenic University NLP
• Model type: Token Classification
• Language(s) (NLP): English
• Finetuned from model: FacebookAI/roberta-base

Uses

<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->

Load model directly

from transformers import AutoTokenizer, AutoModelForTokenClassification

tokenizer = AutoTokenizer.from_pretrained("tabiya/roberta-base-job-ner")
model = AutoModelForTokenClassification.from_pretrained("tabiya/roberta-base-job-ner")

Training Details

Training Data

<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->

More information about the training dataset can be found here

Training Procedure

<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->

The training of this model was done using the HuggingFace token classification tutorial

Training Hyperparameters

• Max Length: 128 <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
• Batch Size: 32
• Learning Rate: 0.0001
• Epochs: 5
• Weight Decay: 0.01

Evaluation

<!-- This section describes the evaluation protocols and provides the results. -->

Testing Data, Factors & Metrics

Testing Data

The training was evaluated on the test set of the training dataset provided by Green et. al.

Metrics

<!-- These are the evaluation metrics being used, ideally with a description of why. -->

Following standard procedures for evaluating entity recognition training the chosen metric was the strict span-F1 measure provided by the seqeval library.

Results

Compute Infrastructure

The hyperparameter search and training was done on the Advanced Research Computing (ARC) of the University of Oxford.

Hardware

All training was performed on V100 GPUs.

Citation

<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->

BibTeX:

TBD