qwen3-8b-htr-vines-lora

Qwen3-8B LoRA — HTR Post-correction for Ramon Viñes Diaries

Fine-tuned LoRA adapters for Handwritten Text Recognition (HTR) post-correction, trained on the manuscript diaries of Catalan pianist Ramon Viñes (1890–1915).

Key Result

The fine-tuned 8B model outperforms a general 70B model by 2.7× — demonstrating that domain-specific fine-tuning is essential for historical manuscript correction.

Model Details

• Base model: Qwen/Qwen3-8B
• Method: LoRA (Low-Rank Adaptation)
• LoRA rank: 16, alpha: 32, dropout: 0.05
• Trainable parameters: 43.6M (0.53% of total)
• Training data: 583 page-level pairs (HTR raw → ground truth)
• Training time: 56 minutes on NVIDIA DGX Spark
• Epochs: 3
• Language: Spanish (primary), French, Catalan

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load base model + LoRA adapters
base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-8B",
    trust_remote_code=True,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)
model = PeftModel.from_pretrained(base_model, "albertdatzira/qwen3-8b-htr-vines-lora")
tokenizer = AutoTokenizer.from_pretrained("albertdatzira/qwen3-8b-htr-vines-lora")

model.eval()

# Correct HTR text
system_prompt = (
    "Ets un expert en post-correcció d'HTR dels diaris de Ramon Viñes. "
    "Corregeix els errors de reconeixement del text manuscrit. "
    "Només corregeix errors d'HTR evidents. No modifiquis l'ortografia "
    "de l'autor, noms propis, ni símbols especials (¶, ¬, ⟦ ⟧)."
)

htr_text = "para el almuerzo (á mediddia). Me hristes son todos estos aniversarios"

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": f"Corregeix el següent text HTR:\n{htr_text}"},
]

text = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs, max_new_tokens=4096, do_sample=False,
        pad_token_id=tokenizer.pad_token_id,
    )

generated = outputs[0][inputs["input_ids"].shape[-1]:]
corrected = tokenizer.decode(generated, skip_special_tokens=True)
print(corrected)
# -> "para el almuerzo (á mediodia). Qué tristes son todos estos aniversarios"

Training Data

The model was trained on 583 page-level pairs from the RV-training repository:

• Input: raw HTR output from the Balakirev model (Kraken/eScriptorium)
• Output: human-validated ground truth transcriptions
• Split: by year (not random) to prevent data leakage
• Filtering: pages with GT < 50 chars or CER > 80% excluded

Limitations

• Best on easy pages: the model achieves 89% page improvement rate on test set pages (CER ~10-12%) but only 20-40% on pages with higher baseline CER.
• Truncation: on some long or error-dense pages, the model generates premature end-of-sequence tokens. A fallback mechanism (revert to original if output is >10% shorter) is recommended.
• Hallucinations: the model occasionally invents corrections for proper nouns and numbers. A CER-based fallback (revert if CER worsens) catches these cases.
• Corpus-specific: trained exclusively on the Viñes diaries. Performance on other HTR corpora is unknown.
• Full-page only: trained on full-page text (~2500 chars). Does not generalise to shorter blocks (distribution shift).

Fallback Mechanism

For production use, always apply a fallback:

1. If the output is >10% shorter than the input → return original (truncation)
2. If ground truth is available and CER worsens → return original

This guarantees zero pages are ever worsened.

Citation

@mastersthesis{perez2026finetuning,
    title={Fine-tuning Large Language Models for HTR Post-correction:
           A Case Study on the Ramon Viñes Manuscript Diaries},
    author={Pérez Datsira, Albert},
    school={Universitat de Lleida},
    year={2026},
    type={Master's Thesis}
}

Links

• Pipeline repository: github.com/albeertito7/vines-htr-correction
• Base model: Qwen/Qwen3-8B

Acknowledgments

• Carles Mateu — thesis supervisor
• Esther Solé i Martí — ground truth management and HTR expertise
• Màrius Bernadó — project coordination
• GReia research group — Universitat de Lleida