gpt2-irish-folk-tune-generator

Try the live demo Space: https://huggingface.co/spaces/Vedant3907/gpt2-irish-folk-tune-generator-space

<video controls width="100%"> <source src="https://huggingface.co/Vedant3907/gpt2-irish-folk-tune-generator/resolve/main/assets/irish-gpt2-model.mp4" type="video/mp4"> </video>

Weekend project demo: an original dataset tune is played first, followed by a GPT-2 generated tune from similar ABC notation context.

GPT-2 Irish ABC Tune Generator

This model is a full fine-tune of openai-community/gpt2 on the sander-wood/irishman dataset, a collection of Irish folk tunes represented in ABC notation.

The model generates symbolic music text, not audio. Generated output can be pasted into an ABC player such as abc.rectanglered.com or ABCjs Editor to hear the tune.

Model Details

• Base model: openai-community/gpt2
• Training method: Full fine-tuning
• Dataset: sander-wood/irishman
• Task: Causal language modeling / ABC notation continuation
• Max sequence length: 512 tokens
• Training hardware: Google Colab T4 GPU
• Training duration: Approximately one epoch
• Validation loss: 0.9962592720985413

What The Model Learns

The training text was formatted as:

<control code>
<ABC notation>

So a prompt can start with a control code, and the model will continue by generating ABC notation.

Example prompt:

S:2 B:8 E:6 B:8

The output should look like ABC music notation with headers and note sequences.

Usage

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "Vedant3907/gpt2-irish-folk-tune-generator"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()

def generate_tune(prompt="S:2 B:8 E:6 B:8\n", max_new_tokens=400):
    inputs = tokenizer(prompt, return_tensors="pt").to(device)
    with torch.no_grad():
        output = model.generate(
            **inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=0.9,
            top_k=40,
            top_p=0.95,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )
    return tokenizer.decode(output[0], skip_special_tokens=True)

print(generate_tune())

Example Prompt

S:2 B:8 E:6 B:8

If the generated output does not include a complete ABC header, add one manually before playing it:

X:1
M:4/4
L:1/8
K:G

Then paste the full ABC text into:

• https://abc.rectanglered.com/
• https://editor.drawthedots.com/
• https://thesession.org/tunes/convert

Evaluation

The model was evaluated on the dataset validation split.

eval_loss: 0.9962592720985413

The final logged training loss was around 1.02, and the validation loss was close to that value, suggesting the model did not obviously overfit during this run.

Training Loss Table

Compact view of the training loss curve. The full logged loss table is available in training_loss_curve.csv.

Limitations

• The model generates ABC notation, not direct audio.
• Some generations may be syntactically invalid ABC.
• Some outputs may need manual cleanup before playback.
• The model may reproduce fragments or patterns from the training data.
• Musical quality varies; sampling multiple outputs and selecting the best one is recommended.

Intended Use

This model is intended for experimentation with Irish folk tune generation, symbolic music modeling, and ABC notation text generation.

It is not intended for claims of originality or commercial music production without additional review for memorization and licensing concerns.

Training Summary

This was trained as a practical free-GPU fine-tuning experiment after the original Karpathy autoresearch training setup proved unsuitable for Google Colab's free T4 GPU. Instead of training from scratch, this model uses GPT-2 as a pretrained base and adapts it to Irish ABC notation.