Downloads · 30 days
11
13% of all-time downloads
Ashybalka/xlm-roberta-taxonomy-healthcare-de
xlm-roberta-taxonomy-healthcare-de is a text classification model from Ashybalka. Use it when you need a label for a piece of text. It is set up for transformers. The card lists the license as apache-2.0.
[](https://opensource.org/licenses/Apache-2.0) [](https://huggingface.co/models?language=de) [](https://huggingface.co/FacebookAI/xlm-roberta-base)
Downloads · 30 days
11
13% of all-time downloads
All-time downloads
82
Public
Repo size
2.5 GB
Likes
0
Public
Click a slice to open those files.
.onnx1.4 GB · 55%
From the Hugging Face model README
Fine-tuned xlm-roberta-base for classifying German healthcare and medical job listings into 11 healthcare subcategories.
Part of the JobBlast taxonomy model family — a system for automated classification of German-language job postings. This model provides fine-grained classification within the Healthcare & Medical top-level category, distinguishing physicians, nursing, therapy, dentistry, pharmacy, lab/diagnostics, emergency services, and several other healthcare professions that the broader main taxonomy collapses into a single bucket.
| Metric | Value |
|---|---|
| Accuracy | 94.50% |
| F1 macro | 87.80% |
| F1 weighted | 94.50% |
Evaluated on a held-out test set of 1,448 German healthcare job listings.
| Category | Precision | Recall | F1 | Support |
|---|---|---|---|---|
| Nursing & Care | 0.992 | 0.968 | 0.980 | 624 |
| Therapy & Rehabilitation | 0.989 | 0.984 | 0.986 | 182 |
| Emergency & Paramedic Services | 0.933 | 1.000 | 0.966 | 14 |
| Dentistry | 0.929 | 1.000 | 0.963 | 104 |
| Pharmacy & Pharma Industry | 0.929 | 0.951 | 0.940 | 41 |
| Medical Doctors | 0.943 | 0.933 | 0.938 | 194 |
| Medical Assistance | 0.868 | 0.937 | 0.901 | 175 |
| Laboratory & Diagnostics | 0.742 | 0.852 | 0.793 | 27 |
| Medical Technology & Devices | 0.864 | 0.731 | 0.792 | 26 |
| Healthcare Management & Administration | 0.833 | 0.676 | 0.746 | 37 |
| Psychology & Psychotherapy | 0.640 | 0.667 | 0.653 | 24 |
The gap between F1 macro (0.878) and F1 weighted (0.945) reflects the underlying class imbalance — the top 7 categories perform at F1 ≥ 0.90, while smaller and semantically harder categories (Psychology, Lab, Medical Technology, Healthcare Management) sit in the 0.65–0.79 range.
| Label | Typical signals in German job ads |
|---|---|
| Medical Doctors | Arzt, Facharzt, Assistenzarzt, Oberarzt, Chefarzt, Hausarzt — all licensed physician roles (approbierte Ärzte, Dr. med.) |
| Nursing & Care | Krankenpfleger, Pflegefachkraft, Altenpfleger, Pflegehelfer, Intensivpflege, Heilerziehungspflege, ambulante Pflege |
| Therapy & Rehabilitation | Physiotherapeut, Ergotherapeut, Logopäde, Sporttherapeut, Heilmittelerbringer |
| Psychology & Psychotherapy | Psychologe, Psychotherapeut, klinische Psychologie, Sozialpädagoge im psychiatrischen Kontext |
| Pharmacy & Pharma Industry | Apotheker, PTA, Medical Science Liaison, Pharmaberater, Drug Development, Regulatory Affairs, klinische Forschung |
| Medical Technology & Devices | Medizintechniker, Biomedizintechnik, medical device engineer, Medizinprodukte, CSSD, Sterilisationsassistent |
| Laboratory & Diagnostics | MTLA, BTA, MTRA, Laborant, Radiologieassistenz, Pathologie, Zytologie, medizinisches Labor |
| Dentistry | Zahnarzt, ZFA (Zahnmedizinische Fachangestellte), Zahntechniker, Kieferorthopädie, Oralchirurgie |
| Emergency & Paramedic Services | Notfallsanitäter, Rettungssanitäter, Rettungsassistent, Paramedic, Leitstelle, Feuerwehr-Sanitäter |
| Medical Assistance | MFA (Medizinische Fachangestellte), OTA (Operationstechnische Assistenz), ATA (Anästhesietechnische Assistenz), Arzthelfer, Praxisassistenz |
| Healthcare Management & Administration | Krankenhausmanager, Qualitätsmanager Klinik, Medizincontrolling, DRG-Kodierung, Healthcare Admin, Patientenaufnahme, Gesundheitsökonom |
Ashybalka/xlm-roberta-taxonomy-healthcare-de/
├── config.json # shared — label map, model config
├── tokenizer.json # shared — fast tokenizer
├── tokenizer_config.json # shared
├── sentencepiece.bpe.model # shared
├── special_tokens_map.json # shared
├── test_metrics.json # evaluation results
├── classification_report.txt # full per-class report
│
├── pytorch/
│ ├── config.json # needed for from_pretrained(subfolder=)
│ └── model.safetensors # GPU inference / fine-tuning (~1.1 GB)
│
├── onnx/
│ └── model.onnx # CPU fp32 inference (~1.1 GB)
│
└── onnx-int8/
└── model_quantized.onnx # CPU INT8 quantized (~280 MB, 3-4× smaller)
[Job Title] [SEP] [Job Description]
text = "Gesundheits- und Krankenpfleger (m/w/d) [SEP] Examinierte Pflegefachkraft " \
"für die Intensivstation, Schichtdienst, abgeschlossene Ausbildung erforderlich."
from transformers import pipeline
clf = pipeline(
"text-classification",
model="Ashybalka/xlm-roberta-taxonomy-healthcare-de",
subfolder="pytorch",
device=0, # GPU; -1 for CPU
)
result = clf("Facharzt für Innere Medizin [SEP] Wir suchen einen Facharzt für unsere Klinik, abgeschlossene Facharztausbildung erforderlich.")
print(result)
# [{'label': 'Medical Doctors', 'score': 0.9712}]
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline
model = ORTModelForSequenceClassification.from_pretrained(
"Ashybalka/xlm-roberta-taxonomy-healthcare-de",
subfolder="onnx"
)
tokenizer = AutoTokenizer.from_pretrained("Ashybalka/xlm-roberta-taxonomy-healthcare-de")
clf = pipeline("text-classification", model=model, tokenizer=tokenizer)
result = clf("Notfallsanitäter [SEP] Einsatz im Rettungsdienst, abgeschlossene Ausbildung als Notfallsanitäter erforderlich.")
print(result)
# [{'label': 'Emergency & Paramedic Services', 'score': 0.9847}]
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import AutoTokenizer, pipeline
model = ORTModelForSequenceClassification.from_pretrained(
"Ashybalka/xlm-roberta-taxonomy-healthcare-de",
subfolder="onnx-int8"
)
tokenizer = AutoTokenizer.from_pretrained("Ashybalka/xlm-roberta-taxonomy-healthcare-de")
clf = pipeline("text-classification", model=model, tokenizer=tokenizer)
For production FastAPI services or environments without transformers:
import json
import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download
from tokenizers import Tokenizer
# download model
path = snapshot_download(
"Ashybalka/xlm-roberta-taxonomy-healthcare-de",
allow_patterns=["onnx/model.onnx", "tokenizer.json", "config.json"]
)
# load
session = ort.InferenceSession(f"{path}/onnx/model.onnx",
providers=["CPUExecutionProvider"])
tokenizer = Tokenizer.from_file(f"{path}/tokenizer.json")
tokenizer.enable_truncation(max_length=510)
tokenizer.no_padding()
with open(f"{path}/config.json") as f:
labels = [v for _, v in sorted(json.load(f)["id2label"].items(), key=lambda x: int(x[0]))]
vocab = tokenizer.get_vocab()
bos, eos = vocab["<s>"], vocab["</s>"]
def classify_healthcare(title: str, description: str) -> dict:
text = f"{title} [SEP] {description}"
encoding = tokenizer.encode(text, add_special_tokens=False)
ids = [bos] + encoding.ids + [eos]
mask = [1] * len(ids)
logits = session.run(None, {
"input_ids": np.array([ids], dtype=np.int64),
"attention_mask": np.array([mask], dtype=np.int64),
})[0][0]
exp = np.exp(logits - logits.max())
probs = exp / exp.sum()
idx = int(np.argmax(probs))
return {"category": labels[idx], "confidence": round(float(probs[idx]), 4)}
print(classify_healthcare(
"Zahnmedizinische Fachangestellte",
"ZFA für unsere moderne Zahnarztpraxis, Stuhlassistenz und Patientenbetreuung, abgeschlossene Ausbildung erforderlich."
))
# {'category': 'Dentistry', 'confidence': 0.9523}
| Parameter | Value |
|---|---|
| Base model | FacebookAI/xlm-roberta-base |
| Training samples | 14,473 German healthcare job listings |
| Labeling | LLM consensus (3-model voting: Qwen3-4B + Gemma-2-9B + Llama-3.1-8B) |
| Agreement filter | 2/3 or 3/3 required |
| Soft labels | Vote distribution used as soft targets (e.g. [Lab, Lab, MedTech] → {Lab: 0.67, MedTech: 0.33}) |
| Max length | 512 tokens |
| Learning rate | 2e-5 |
| Epochs | 5 (early stopping, best epoch: 4) |
| Class weighting | Balanced |
| Category | Count | Share |
|---|---|---|
| Nursing & Care | 6,238 | 43.1% |
| Medical Doctors | 1,942 | 13.4% |
| Therapy & Rehabilitation | 1,817 | 12.6% |
| Medical Assistance | 1,747 | 12.1% |
| Dentistry | 1,038 | 7.2% |
| Pharmacy & Pharma Industry | 407 | 2.8% |
| Healthcare Management & Administration | 373 | 2.6% |
| Laboratory & Diagnostics | 271 | 1.9% |
| Medical Technology & Devices | 254 | 1.8% |
| Psychology & Psychotherapy | 242 | 1.7% |
| Emergency & Paramedic Services | 144 | 1.0% |
The 43.3× imbalance ratio between Nursing & Care and Emergency & Paramedic Services is handled through balanced class weights at training time. Despite the imbalance, even the smallest class (Emergency, 144 samples) achieves F1 0.966 — soft-label training and class weighting compensate effectively when the LLM consensus on the class is clean.
Healthcare & Medical into this classifier.| Model | Categories | Use case |
|---|---|---|
| xlm-roberta-taxonomy-main-de | 21 top-level | Industry / field classification |
| xlm-roberta-taxonomy-it-de | 14 IT subcategories | Detailed IT role classification |
| xlm-roberta-taxonomy-seniority-de | 5 grades | Experience-level classification, orthogonal to industry |
| xlm-roberta-taxonomy-healthcare-de (this model) | 11 healthcare subcategories | Fine-grained classification within healthcare jobs |
A typical JobBlast pipeline: the main model assigns the top-level category. Listings tagged as Healthcare & Medical are passed to this model for fine-grained role classification (Nursing, Doctors, Dentistry, etc.).