xlm-roberta-xl-german-skill-classifier

Model Details

Model Description

• Developed by: Mahir Yilmaz TURK
• Funded by Anonymous Client
• Model type: XLM-Roberta-XL
• Language(s) (NLP): German
• Finetuned from model XLM-Roberta-XL

Uses

This model is fine tuned with the intention to classify a given German Task/Skill name into one of the 5 classes:

<code>label_names = ["cognitive", "administrative", "social", "manual", "digital"]</code>

Direct Use

import torch
import numpy as np
from transformers import AutoTokenizer, AutoModelForSequenceClassification, AutoConfig

MODEL_NAME = "mhrtrk/xlm-roberta-xl-german-skill-classifier"
label_names = ["cognitive", "administrative", "social", "manual", "digital"]
MAX_LENGTH = 128

config = AutoConfig.from_pretrained(
    MODEL_NAME,
    num_labels=len(label_names),
    problem_type="multi_label_classification",
)
tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)
model = AutoModelForSequenceClassification.from_pretrained(MODEL_NAME, config=config)

def predict(text):
    inputs = tokenizer(
        text,
        truncation=True,
        padding="max_length",
        max_length=MAX_LENGTH,
        return_tensors="pt"
    )
    
    outputs = model(**inputs)
    logits = outputs.logits
    
    probs = torch.sigmoid(logits)
    
    preds = (probs > 0.5).numpy().astype(int)
    
    predicted_labels = [label for label, flag in zip(label_names, preds[0]) if flag == 1]
    return predicted_labels, probs.detach().numpy()

example_text = "d) Kraftübertragungssysteme, insbesondere Schaltgetriebe,\nAutomatikgetriebe und Allradsysteme, instand\nsetzen"
predicted_labels, probabilities = predict(example_text)
print("Predicted Labels:", predicted_labels)
print("Probabilities:", probabilities)

Metrics

<pre> Epoch Training Loss Validation Loss F1 Exact Match 1 0.221100 0.194390 0.853914 0.727347 2 0.164300 0.172580 0.863857 0.745306 3 0.140200 0.181491 0.865734 0.744490 4 0.118300 0.185243 0.865450 0.752653 </pre>

Model Card Authors

Mahir Yilmaz TURK

Model Card Contact

[email protected]