chilean-spanish-judgment-subtypes-bert-tulio

Judgment Subtypes — TULIO token classifier (Chilean Spanish)

This model tags the five Judgment subtypes of Appraisal Theory in Systemic Functional Linguistics: normality, capacity and tenacity under Social Esteem, veracity and propriety under Social Sanction. It is a TULIO encoder with a token-classification head, fine-tuned on the Chilean Spanish Attitude Corpus, and it is the transformers entry for Judgment Subtypes in Table 3 of the accompanying paper. The released weights are the seed-42 run whose predictions produce the value printed there.

Content warning. The training corpus was collected around episodes of political conflict in Chile and contains insults, hate speech and threats. The model was fitted on that language and will reproduce its distribution.

Labels

The model emits flat token labels over the classes capacity, normality, propriety, tenacity, veracity, plus O. A span is a maximal contiguous run of the same non-O class.

Usage

from transformers import pipeline

tagger = pipeline(
    "token-classification",
    model="jorgeortizfuentes/chilean-spanish-judgment-subtypes-bert-tulio",
    aggregation_strategy="simple",
)
print(tagger("Ese diputado es un sinvergüenza y el proyecto es lamentable."))

The labels are flat token classes without BIO prefixes, so a span is a maximal contiguous run of the same non-O class; aggregation_strategy="simple" reproduces that grouping. Training and evaluation used pre-tokenized words (is_split_into_words=True, max_length=512), with the label of a word on its first sub-token. To reproduce the reported scores, tokenize the corpus tokens field the same way and read the prediction of each word's first sub-token.

Evaluation

Scores are strict span level: a predicted span counts only when its class and both boundaries match the gold span. They come from the 382-text test split of the corpus, recomputed from the saved predictions of this run.

The three retrainings change only the seed. Their spread describes run-to-run variation of the recipe, not the uncertainty of this checkpoint, and it is not a significance test. The released weights are the seed-42 run, so its F1 is the one this file reproduces.

Every score stays below expert agreement. The task is not solved.

Per class

Training data

The corpus holds 2,546 Chilean Spanish texts annotated by three linguists trained in SFL, split 1,782 / 382 / 382. It is gated: the source texts are public posts whose authors did not consent to redistribution, so access is granted for non-commercial research under the corpus LICENSE. The weights released here are not gated; they do not redistribute the texts.

The corpus is dominated by tweets (2,420 of 2,546 texts). Letters to the editor, opinion columns and consumer complaints are present in small numbers.

Training configuration

Base model: dccuchile/tulio-chilean-spanish-bert (DOI 10.57967/hf/1846).

Validation scores of this run: accuracy 0.9060, f1 0.3309, precision 0.3185, recall 0.3442.

Intended use

Research on Appraisal, span-level sequence labeling and evaluative language in Chilean Spanish. The fixed splits allow comparison with the paper.

It is not fit for moderating users, profiling individuals, making decisions about people, or as a general-purpose sentiment classifier. Attitude analysis identifies evaluative language and its category; it does not identify who is right, and a Judgment label is not a finding about the person being judged.

Limitations

Scores are far below what sequence labeling reaches on more established tasks, and below the agreement of the experts who annotated the data. Judgment Subtypes is the harder of the two tasks for every architecture tested.

The model was trained on Chilean Spanish from 2010–2022, mostly Twitter. Under Twitter-only training, both supervised architectures in the paper score lower outside Twitter, particularly on Judgment Subtypes; cross-genre transfer is not established. Performance on other varieties of Spanish is untested.

Provenance

Citation

@inproceedings{ortizfuentes-attitude-2026,
  title     = {Attitude Analysis in Systemic Functional Linguistics: A New Corpus and Benchmark Task},
  author    = {Ortiz-Fuentes, Jorge and Bravo-Marquez, Felipe and Quiroz, Beatriz},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026},
  year      = {2026},
  note      = {To appear}
}

Cite TULIO as well when you use this model; the encoder is dccuchile/tulio-chilean-spanish-bert.

License

The fine-tuned weights are released under CC BY 4.0, the license declared by the TULIO encoder they derive from. The training corpus keeps its own license and access conditions.