thai-job-ner-phayathaibert

Thai Job NER — Fine-tuned PhayaThaiBERT (v2: Mixed Training)

Named Entity Recognition model for extracting structured HR data from informal Thai job postings (e.g., Facebook groups, Line chats). Fine-tuned from PhayaThaiBERT (~122M params).

v2 update: Trained on style-matched synthetic + real-world data with upsampling. Real-world F1 improved from 0.143 → 0.975.

Model Description

This model extracts 7 entity types from Thai job-related text:

Usage

from transformers import AutoTokenizer, AutoModelForTokenClassification, pipeline

model_name = "chayuto/thai-job-ner-phayathaibert"
ner = pipeline("ner", model=model_name, aggregation_strategy="simple")

text = "รับสมัครคนดูแลผู้สูงอายุ ย่านสีลม เงินเดือน 18,000 บาท โทร 081-234-5678"
results = ner(text)
for entity in results:
    print(f"{entity['entity_group']}: {entity['word']} ({entity['score']:.2%})")

Training

• Base model: clicknext/phayathaibert (CamemBERT architecture, ~122M params, XLM-R-derived vocabulary)
• Training data: 619 posts (510 style-matched synthetic + 54 real-world posts upsampled 5x), silver labels from GPT-4o, fuzzy-aligned to IOB2
• Real data proportion: 34.6% (key factor for real-world performance)
• Hardware: Apple Silicon MPS backend, FP32
• Hyperparameters: LR=3e-5, warmup=0.1, batch=2, grad_accum=4, 10 epochs, gradient checkpointing, frozen embeddings
• Training time: ~35 min

Data Pipeline

Raw Thai text + GPT-4o entity extractions → fuzzy alignment with rapidfuzz + pythainlp TCC boundary snapping → subword token mapping via offset_mapping → IOB2-formatted HuggingFace Dataset.

Training Strategy

Style-matched synthetic data (v3) was generated to mimic real-world post characteristics: varied formatting, code-switching, informal language. Real-world posts were upsampled 5x to achieve a 20-35% real data proportion — the sweet spot identified through systematic experiments (see below).

Evaluation

Real-World Test Set (8 held-out real posts, 98 entities)

Real-World Per-Entity F1

5 of 7 entity types achieve perfect F1 on real-world data.

Mixed Test Set (78 examples)

Experiment History: Closing the Real-World Gap

Key finding: real data proportion is the #1 factor. Style-matched synthetic data is 3x more efficient than generic synthetic data.

Links

• Model: chayuto/thai-job-ner-phayathaibert
• WangchanBERTa variant: chayuto/thai-job-ner-wangchanberta
• Dataset: chayuto/thai-job-ner-dataset
• Source Code: github.com/chayuto/thai-job-nlp-ner

Limitations

• Real-world evaluation is based on 8 held-out posts (98 entities) — confidence intervals are wide
• EMPLOYMENT_TERMS remains the weakest entity (F1=0.880) — boundary ambiguity in schedule/contract terms
• Embeddings were frozen during training (MPS memory constraint) — unfreezing on a larger GPU may yield further gains
• 256 token max sequence length (covers >95% of real posts)
• Larger model file size due to 248K vocabulary (vs WangchanBERTa's 25K)

Technical Notes

• FP16 is broken on MPS — always use FP32 for Apple Silicon training
• PhayaThaiBERT's 248K vocab (XLM-R-derived) requires frozen embeddings + gradient checkpointing to fit on 18GB MPS
• Uses offset_mapping for tokenizer-agnostic subword-to-character alignment
• Thai Character Cluster (TCC) boundary snapping prevents Unicode grapheme splitting during alignment
• Real data upsampling (5-10x) is a simple, effective technique for low-resource scenarios

License

MIT