fftuniba-cruciverbit-it5-efficient-small-el32

FFT-UniBa Cruciverb-IT IT5-Efficient-Small

This is the model released by the FFT-UniBa team (University of Bari Aldo Moro) for the Cruciverb-IT shared task at EVALITA 2026.

It is it5-efficient-small-el32 fine-tuned for Italian crossword clue answering: given a clue and its expected answer length, it generates a ranked list of candidate solutions of that exact length. The same model powers the team's Sub-task 2 crossword solver, where its candidates are fed to a CSP/backtracking search (see the official code repository).

The key ingredient is a Dual-Constraint mechanism: 28 special tokens ([SL = k] / [EL = k] for k = 1..28) are added to the tokenizer and the target length is embedded both at the start of the input and in the output, so the model internalizes the length constraint instead of relying on post-hoc filtering.

Model Details

• Developed by: Andrea Porcelli, Filippo di Gravina, Emanuele Fontana, Mattia Curri, Francesco Damiano di Gregorio (FFT-UniBa)
• Model type: encoder-decoder Transformer (T5 / Efficient T5 architecture), fine-tuned with supervision
• Language(s): Italian (it)
• License: Apache 2.0 (same as the base model)
• Finetuned from: gsarti/it5-efficient-small-el32
• Repository: https://github.com/mattiacurri/fftuniba-cruciverbit
• Paper: FFT-UniBa at Cruciverb-IT: Special Length Tokens and CSP for Italian Crossword Solving (CEUR-WS Vol-4195, EVALITA 2026)

Revisions

Both revisions share the same architecture, tokenizer and special tokens; only the weights differ. Load the second one with revision="grid-solver-22epochs".

Uses

Direct Use

Answer Italian crossword clues, given the expected answer length. The model expects the input format used during fine-tuning:

indovinello: <CLUE> [SL = <LENGTH>]

and generates [SL = <LENGTH>] <ANSWER> [EL = <LENGTH>], where the length tokens are stripped when decoding with skip_special_tokens=True.

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

repo = "fgoose180/fftuniba-cruciverbit-it5-efficient-small-el32"
# for the grid-solving variant: revision="grid-solver-22epochs"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

clue, length = "Umbro d'un capoluogo", 7
prompt = f"indovinello: {clue} [SL = {length}]"

inputs = tokenizer(prompt, return_tensors="pt", truncation=True, max_length=128)
inputs.pop("token_type_ids", None)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_length=32,
        num_beams=100,
        num_return_sequences=100,
        output_scores=True,
        return_dict_in_generate=True,
        early_stopping=True,
    )

candidates = [c.strip() for c in tokenizer.batch_decode(out.sequences, skip_special_tokens=True)]
# "constrained" mode: keep only candidates matching the target length (recommended, ~+0.03 MRR)
constrained = [c for c in candidates if len(c) == length]
print(constrained[:10])

The sequences_scores returned by generate are the sequence log-probabilities used to rank the candidates (lower is better) in the original pipeline.

Downstream Use

The model was used as the candidate generator of a CSP solver: the top-K candidates per clue (plus a small number K' of dictionary words) form the domains of a depth-first search with backtracking that fills the whole crossword grid while satisfying crossing constraints. See the official repository for the solver and the exact hyperparameters.

Out-of-Scope Use

• General-purpose Italian text generation, translation, summarization or any task the model was not fine-tuned for.
• Solving crosswords in languages other than Italian.
• Any user-facing application without human review: outputs are candidate lists, not verified answers.

Bias, Risks, and Limitations

• Bias: the base model was pretrained on gsarti/clean_mc4_it, a web-scraped Italian corpus, so it can reproduce and amplify social biases and stereotypes present in that data. The fine-tuning data (crossword clues, mostly from older Italian puzzles) adds a further cultural and temporal skew.
• Limitations: the model is a clue answerer, not a crossword solver: it does not check grid consistency on its own. It can produce fluent but wrong words, and quality degrades for very long answers and for rare proper nouns. Without the length token the generation is unconstrained and much less precise.
• Data terms: the fine-tuning corpus cruciverb-it/evalita2026 is released under copyrighted-research-purposes; review those terms before any use beyond research.
• Recommendations: always post-filter candidates by the target length and rank them with beam-search scores; for grid filling, validate the solution with the crossing constraints (e.g. the CSP solver) instead of trusting the model output directly.

How to Get Started with the Model

See the snippet in Direct Use. The model is a standard transformers seq2seq model, so it also works with the text2text-generation pipeline.

Training Details

Training Data

• cruciverb-it/evalita2026 — Cruciverb-IT Sub-task 1 training split (Italian crossword clues with answers and lengths).
• mik3ml/italian-dictionary — dictionary augmentation: ~313,000 definition/word pairs, preprocessed exactly like the crossword clues, to improve vocabulary coverage.

Each example is formatted as indovinello: <CLUE> [SL = L] → [SL = L] <ANSWER> [EL = L].

Training Procedure

Fine-tuning of it5-efficient-small-el32 with the Hugging Face Trainer. Hyperparameters reported in the paper:

The 28 length tokens ([SL = k]/[EL = k], k = 1..28) are added to the tokenizer and their embeddings are randomly initialized; the embedding matrix grows from 32128 to 32156 rows. Hardware / training time were not tracked.

Speeds, Sizes, Times

142M parameters in fp32 (569 MB model.safetensors). Checkpoints were saved every epoch.

Evaluation

Testing Data, Factors & Metrics

• Testing data: Cruciverb-IT 2026 test set — 20,821 clues (Sub-task 1) and 50 grids / 1,582 clues (Sub-task 2).
• Metrics: Accuracy@1, Accuracy@10 and MRR for clue answering; character accuracy, word accuracy and full-grid accuracy for grid filling.
• Factors: answer length (1–28 characters) heavily influences difficulty: accuracy is close to 100% for 5-letter answers and drops for long ones.

Results

Sub-task 1 — clue answering (test set, official submitted scores; constrained = candidates filtered by length):

Sub-task 2 — crossword grid filling (test set, 600 candidates per clue, with dictionary fallback):

Summary

The length-token fine-tuning is what makes the difference: with the same data and base model, removing it costs ~0.10 Acc@1, and combining dictionary augmentation with length tokens gives the best configuration. Independently re-checked on a 1,500-clue sample of the test set, this checkpoint reproduces the submitted scores (Acc@1 0.561 / Acc@10 0.736 / MRR 0.624).

Environmental Impact

Not tracked.

Technical Specifications

• Architecture: T5ForConditionalGeneration (efficient T5 small, EL32 encoder), d_model 512, 32 encoder / 6 decoder layers, 8 heads, vocabulary 32,156 tokens (32,128 + 28 length tokens).
• Hardware requirements: runs on CPU for small batches; a GPU is recommended for beam search with hundreds of beams.
• Software: transformers >= 4.56, tokenizers, sentencepiece not required (fast tokenizer).

Citation

@inproceedings{porcelli2026fft,
  title={FFT-UniBa at Cruciverb-IT: Special Length Tokens and CSP for Italian Crossword Solving},
  author={Porcelli, Andrea and Di Gravina, Filippo and Fontana, Emanuele and Curri, Mattia and Di Gregorio, Francesco Damiano},
  booktitle={Proceedings of the Ninth Evaluation Campaign of Natural Language Processing and Speech Tools for Italian. Final Workshop (EVALITA 2026), CEUR-WS, Bari, Italy},
  year={2026}
}

If you use the base model, please also cite IT5:

@inproceedings{sarti-nissim-2024-it5-text,
  title = "{IT}5: Text-to-text Pretraining for {I}talian Language Understanding and Generation",
  author = "Sarti, Gabriele and Nissim, Malvina",
  booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
  month = may,
  year = "2024",
  address = "Torino, Italia",
  publisher = "ELRA and ICCL",
  url = "https://aclanthology.org/2024.lrec-main.823",
  pages = "9422--9433"
}

Model Card Contact

For questions about the model or the paper, open an issue on the official repository. This copy is hosted on the Hugging Face account of a team member.