tei-bibl-annotator-citext

TEI Bibliographic Annotator

A LoRA fine-tune of Qwen/Qwen2.5-3B-Instruct, merged back into the base weights, that tags a plain-text bibliographic citation with inline TEI XML markup. It reproduces the input text exactly, inserting tags around the spans that carry bibliographic structure (author, title, date, publisher, and so on) — nothing is paraphrased or normalized away. The output is just the tagged citation: no XML declaration, no wrapping element.

Source & training pipeline: github.com/cboulanger/tei-annotation-model · Live demo: cmboulanger/tei-bibl-annotator-demo

The training citations come from footnotes in legal-sociology scholarship (mostly German, some English), where citations are frequently given in short-form or anaphoric form ("Id.", "ibid.", a bare case name introduced earlier) and are mixed in with the author's own prose — harder than the clean one-per-line entries of a reference list.

Example:

input:  Scheingold, The Politics of Rights (1974).
output: <author><persName><surname>Scheingold</surname></persName>,</author>
        <title level="a">The Politics of Rights</title> <date>(1974).</date>

License

This checkpoint is for research use only. The base model, Qwen/Qwen2.5-3B-Instruct, is released under Alibaba's Qwen Research License, which restricts commercial use. Because the LoRA adapter is merged directly into the base weights, this checkpoint is a derivative work and inherits those terms. See the linked license text for the exact conditions.

How to use

The model expects the same chat prompt used during training and evaluation: a fixed system instruction plus the raw citation as the user turn. Generation is greedy (do_sample=False), max_new_tokens=1024.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "cmboulanger/tei-bibl-annotator"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

INSTRUCTIONS = (
    "Tag this bibliographic citation with inline TEI XML markup, exactly "
    "reproducing the input text with tags inserted around the parts that "
    "carry structure (author, title, date, etc). Output only the tagged "
    "citation, with no XML declaration or wrapping element."
)


def build_messages(input_text: str) -> list[dict]:
    return [
        {"role": "system", "content": INSTRUCTIONS},
        {"role": "user", "content": input_text},
    ]


tokenizer = AutoTokenizer.from_pretrained(REPO)
model = AutoModelForCausalLM.from_pretrained(REPO, dtype=torch.bfloat16).to(DEVICE)
model.eval()

input_text = "Scheingold, The Politics of Rights (1974)."
inputs = tokenizer.apply_chat_template(
    build_messages(input_text), add_generation_prompt=True,
    tokenize=True, return_dict=True, return_tensors="pt",
).to(DEVICE)

with torch.inference_mode():
    generated = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
completion_ids = generated[:, inputs["input_ids"].shape[1]:]
print(tokenizer.batch_decode(completion_ids, skip_special_tokens=True)[0].strip())

Results

First training pilot: LoRA fine-tune (rank 16, all attention + MLP projections) for 3 epochs over the 9,050-record train set; training loss dropped to 0.028. Scored on the full 1,207-record held-out dev set:

These are a lightweight sanity signal — is the output well-formed, and how often is a citation exactly right — not a full per-field or per-tier evaluation.

Training data

99 TEI documents, each a <listBibl> of individually tagged <bibl> citation records extracted from footnotes. After filtering one degenerate record, 11,467 records remain, split by whole document (a document's citations are never split across sets) at roughly an 80/10/10 ratio:

The split is stratified so the rare legal tier (citations containing a case reference) appears in the dev and test sets, not only in train.