intent_128k_v13

Deprem Niyet Sınıflandırma (Dataset v1.3, BERT 128k)

Alakasız sınıfı atılarak eğitildi.

Eval Results

                 precision    recall  f1-score   support

        Lojistik       0.83      0.86      0.84        22
Elektrik Kaynagi       0.71      0.95      0.81        39
  Arama Ekipmani       0.72      0.80      0.76        82
          Cenaze       0.50      0.33      0.40         3
           Giysi       0.79      0.96      0.87        91
  Enkaz Kaldirma       0.99      0.95      0.97       601
          Isinma       0.75      0.90      0.82       112
         Barınma       0.98      0.95      0.96       292
         Tuvalet       0.83      1.00      0.91         5
              Su       0.80      0.85      0.83        39
           Yemek       0.94      0.95      0.94       138
          Saglik       0.80      0.85      0.83        75

       micro avg       0.90      0.93      0.92      1499
       macro avg       0.80      0.86      0.83      1499
    weighted avg       0.91      0.93      0.92      1499
     samples avg       0.94      0.95      0.94      1499

Reproducibility icin trainer arg'lari:

TrainingArguments(
    fp16=True,
    evaluation_strategy = "steps",
    save_strategy = "steps",
    learning_rate=5.1058553791201954e-05,
    per_device_train_batch_size=batch_size,
    per_device_eval_batch_size=batch_size*2,
    num_train_epochs=4,
    load_best_model_at_end=True,
    metric_for_best_model="macro f1",
    logging_steps = step_size,
    seed = 42,
    data_seed = 42,
    dataloader_num_workers = 0,
    lr_scheduler_type ="linear",
    warmup_steps=0,  
    weight_decay=0.06437697487126866,  
    full_determinism = True,
    group_by_length = True
)

Threshold: Best Threshold: 0.52