irish-abc-qwen2-5-lora

Model Card for eoinedge/irish-abc-qwen2-5-lora

This adapter is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct. It was trained by Eoin Jordan to improve ABC notation continuation for Irish traditional-style music generation.

Model description

eoinedge/irish-abc-qwen2-5-lora is a LoRA adapter that steers the base Qwen2.5 instruction model toward generating valid ABC music notation in an Irish traditional style. The adapter was trained using parameter-efficient fine-tuning with PEFT and the Hugging Face TRL SFT framework.

Usage

Load the base model and adapter together to generate ABC continuations:

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-0.5B-Instruct",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
    "eoinedge/irish-abc-qwen2-5-lora",
    trust_remote_code=True,
    use_fast=True,
)
model = PeftModel.from_pretrained(base_model, "eoinedge/irish-abc-qwen2-5-lora")

prompt = "K:D\n|:A2FA DAFA|"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=180)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training data

The adapter was trained on a curated ABC continuation dataset stored in this repository:

• Training set: data/processed/train_10k.jsonl
• Validation set: data/processed/val_1k.jsonl

The dataset is derived from public-domain and traditional ABC sources, formatted for supervised continuation training.

Training setup

• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Adapter type: LoRA (peft)
• Sequence length: 512 tokens
• Epochs: 1
• Learning rate: 2e-4
• Batch size: 1
• Gradient accumulation steps: 16
• LoRA rank: 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• Gradient checkpointing: enabled
• Learning rate scheduler: cosine
• Warmup steps: 100
• Precision: fp16
• Trainer: trl.SFTTrainer

The training configuration is defined in configs/qwen2_5_0_5b_lora_mac.yaml and the fine-tuning script is scripts/train_lora.py.

Evaluation

• Local evaluation was run on 17 May 2026 using the repository's generation tooling and the adapter artifact in outputs/adapters/irish-abc-qwen2_5-0_5b-lora-mac.
• Deterministic inference (do_sample=False) with the current local MPS environment produced continuations dominated by repeated punctuation characters, indicating unstable adapter generation behavior for the tested ABC prompt.
• Sampled generation with the current local setup also encountered numerical instability during prediction, raising an invalid logits error (probability tensor contains either inf, nan or element < 0).
• These results are preliminary; further prompt tuning, adapter refinement, or a different inference runtime is recommended before relying on this adapter for production ABC generation.

Limitations

• This is an adapter, not a standalone full model; it requires the base Qwen model.
• Generated ABC should be reviewed for musical correctness and notation validity.
• The adapter may still produce invalid measures or non-ABC text in some cases.
• Do not use this model to generate copyrighted music without permission.

Citation

If you use this adapter, cite the adapter itself:

@software{jordan2026irishabcqwen,
  author = {Jordan, Eoin},
  title = {Irish ABC Qwen2.5 LoRA Adapter},
  year = {2026},
  url = {https://huggingface.co/eoinedge/irish-abc-qwen2-5-lora}
}

The adapter uses the TRL and PEFT libraries. Use their current project citation files when citing those dependencies.