quote-detection__roberta-base-sentence-v2

RoBERTa-based sentence-level classification model. Similar to alex2awesome/quote-detection__roberta-base-sentence, except it's trained on a lot more data.

Scores this accuracy on the original gold-label training dataset mentioned in: https://arxiv.org/pdf/2305.14904.pdf

{ "eval_samples": 86, "test_f1": 0.8056872037914692, "test_loss": 0.4573868425250922, }

{ "DIRECT QUOTE_f1": 0.9284126278345932, "full_f1": 0.8055235903337169, "BACKGROUND_f1": 0.770718232044199, "NO QUOTE_f1": 0.0, "INDIRECT QUOTE_f1": 0.8142250530785562, "PUBLISHED WORK_f1": 0.8680851063829788, "STATEMENT_f1": 0.9477124183006536, "PRESS REPORT_f1": 0.9457364341085273, "DECLINED COMMENT_f1": 0.9444444444444444, "SOCIAL MEDIA POST_f1": 0.8936170212765957, "PROPOSAL/ORDER/LAW_f1": 0.5616438356164384, "PRICE SIGNAL_f1": 0.7272727272727273, "NARRATIVE_f1": 0.8833333333333333, "DIRECT OBSERVATION_f1": 0.4324324324324324, "COMMUNICATION_f1": 0.9672131147540983, "PUBLIC SPEECH_f1": 0.983050847457627, "VOTE/POLL_f1": 0.8205128205128205, "COURT PROCEEDING_f1": 0.9491525423728813 }