teaching-transformers-classical-italian-sonnets

Teaching Transformers to Write Classical Italian Sonnets

This public release package contains nine artifacts from a controlled research study of literary adaptation of the existing Minerva 7B Instruct model, in two generations. The first generation adapted the model to classical Italian sonnets. The second generation, added in 2026-09, solves measurable sonnet form through an explicit plan-then-poem decomposition, and reports the remaining coherence limit honestly.

The three stages form one sequential lineage. They ultimately begin from sapienzanlp/Minerva-7B-instruct-v1.0 at revision d1fc0f0e589ae879c5ac763e0e4206a4d14a3f6d. The models were not trained from scratch. The DPO adapter must be attached to the stage3 subfolder in this repository; it is not compatible with a generic Minerva checkpoint.

Loading

Load each full model by naming its subfolder:

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "LPM93/teaching-transformers-classical-italian-sonnets"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder="stage3")
stage3 = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder="stage3",
    torch_dtype="auto",
    device_map="auto",
)

Load the adapter only after explicitly loading its exact Stage-3 base:

from peft import PeftModel

dpo_model = PeftModel.from_pretrained(
    stage3,
    repo_id,
    subfolder="dpo_adapter",
)

Use subfolder="stage1" or subfolder="stage2" for the earlier full-model states. PEFT's adapter_config.json records the repository ID but has no standard field for a base-model subfolder, so automatic adapter-only loading is not supported: load stage3 explicitly as shown above.

2026-09 follow-up — repair-free sonnet form

The first generation produced 0 of 4,976 fully form-valid outputs: the rhyme scheme did not hold. The follow-up measured the gap with a purpose-built Italian prosody checker (metre, rhyme, scheme, fourteen-line structure) and a rhyme lexicon, then solved form by decomposition:

1. A plan generator writes a rhyme plan (a fourteen-letter scheme plus one ending word for lines 2-14). It reaches 0.921 valid plans on a 240-plan evaluation grid, against 0/240 for the unadapted baseline.
2. A poem writer writes the sonnet to that plan, without any repair.

Metrics are checker-based: scheme, fourteen lines, hendecasyllable lines, and rhyme. Across the distilled pipeline, 7.3 of 14 lines are accepted hendecasyllables and 0.93 fail; the memorization screen against the 16,298 training sonnets finds zero copied lines.

Coherence is the limit. A calibrated two-judge AI panel scored the composed poems 2.917 and 2.725 of 5, against 2.875 and 2.750 before distillation, while the teacher sonnets themselves scored 4.12 and 4.47. Form transfers; coherence does not. The checker does not measure grammar, meaning, or literary quality.

Merging the second-generation adapters

The adapters attach to a merged chain, in this order:

# plan step: stage3 -> dpo_adapter -> plan_following_adapter -> plan_generator
# poem step: ... -> plan_follower_v2  (clean)  or  poem_follower_distilled (best form)

The plan is generated as text (Schema: ABBAABBACDECDE, then numbered ending words), then passed to the poem writer with the opening line. Full evidence, reports, and the checker live in the project repository: https://github.com/LeonardoPaccianiMori/portfolio-transformer-poetry

Synthetic teacher data and its licence basis

poem_follower_distilled was trained in part on sonnets generated by the third-party DeepSeek Open Platform model deepseek-chat on 2026-09-15. The DeepSeek Open Platform Terms of Service (release 2026-04-22, effective 2026-04-29), section 4.2, state that the user "may apply the Inputs and Outputs of the Services to a wide range of use cases, including personal use, academic research, derivative product development, training other models (such as model distillation), etc."; section 4.2(2) assigns rights in the Outputs to the user. The project recorded the provider, model, date, volumes, filtering, and caveats (PRC governing law, the ban on implying DeepSeek endorsement, AI-content disclosure duties, and export-control rules). No synthetic text or teacher output is distributed. See poem_follower_distilled/TRAINING_CONTENT_SUMMARY.md.

Result boundary

The first-generation sealed test supports a narrow automatic surface/completion gain from AI-judged DPO, not broad literary quality. Human/AI calibration was only 12/20 and failed its gate. Both Stage 3 and DPO produced 0/100 strict-good outputs in the frozen blind review. The 2026-09 follow-up then reached 0.6250-0.6917 composed scheme validity without repair through plan-then-poem decomposition, while coherence stayed at about 2.7-2.9 of 5 under the AI judges. Fourteen-line output remains decoder-controlled. These artifacts are research evidence, not reliable poets, human-aligned systems, or production language models.

Rights and training-data boundary

Hugging Face metadata uses CC BY-NC 4.0 only for copyright and similar rights Leonardo Pacciani-Mori holds, if any, in his original model modifications. Rights independently received in the pinned Minerva parent retain the parent's Apache-2.0 designation. Leonardo-owned model-card prose and aggregate documentation retain CC BY 4.0 where identified.

No corpus text, token streams, prompts, openings, generations, preferences, votes, annotations, mappings, optimizer state, or training logs are included. Source disclosure does not grant corpus-redistribution rights or create one package-wide data license. Read each subfolder's RIGHTS_SCOPE.md, NOTICE.md, TRAINING_CONTENT_SUMMARY.md, and lineage.json before reuse.

The release uses the unresolved assumption that training-data licenses do not govern the weights. If incompatible ShareAlike terms are determined to attach, distribution is not authorized under this structure. The repository is provided without a warranty of title or non-infringement, subject to the included license texts.

Ownership and AI contribution

The project was conceived, directed, and reviewed by Leonardo Pacciani-Mori, who set the goals, gates, and publication decisions. AI assistants assisted design, implementation, execution, and analysis under his direction.

Project source, evidence, and full limitations: https://github.com/LeonardoPaccianiMori/portfolio-transformer-poetry