nepali-folk-acestep-1.5-lora

Nepali Folk ACE-Step 1.5 LoRA

A LoRA adapter for ACE-Step 1.5, fine-tuned for Nepali folk-style music generation.

The adapter specializes ACE-Step toward acoustic Nepali folk characteristics, particularly instrumentation involving Madal, Sarangi, and Bansuri.

It was developed as the music-generation component of an undergraduate final-year project on end-to-end Nepali folk music generation.

Note: This repository contains the LoRA adapter only. The ACE-Step/Ace-Step1.5 base model is required separately.

Quick Overview

Role in the Project

This model is the second stage of the final generation pipeline.

User Song Request
        ↓
Gemma-3-4B Nepali Lyrics LoRA
        ↓
Generated Nepali Lyrics
        ↓
ACE-Step 1.5 Turbo
+ Nepali Folk LoRA
        ↓
Generated Nepali Folk Song

The lyric-generation model creates Nepali Devanagari lyrics first.

Those lyrics are then supplied to ACE-Step together with a music description. This LoRA adapts the audio-generation stage toward the Nepali folk style represented in the training data.

---

Intended Use

This adapter is intended for:

• experimental Nepali folk music generation
• academic research
• lyrics-conditioned music generation
• folk-style instrumental adaptation
• experimentation with traditional Nepali instrumentation
• parameter-efficient fine-tuning research

The training material particularly emphasized:

• Madal
• Sarangi
• Bansuri

---

Training Data

The project initially collected 1,163 WAV audio files.

After dataset preparation and selection, the final LoRA training run used:

400 preprocessed training tensors

The selected training material focused primarily on Nepali folk-style audio and traditional instrumentation.

The dataset was used for the associated academic project with the required permissions obtained for its use.

---

Training Method

The ACE-Step base model was kept frozen while LoRA parameters were trained on selected transformer modules.

This allowed the model to be adapted without updating the full set of approximately 2.4 billion base-model parameters.

LoRA Configuration

Training Configuration

---

Final Inference Configuration

The final project pipeline used:

These settings were selected during project experimentation and used for the final song-generation pipeline.

---

Tesla T4 Numerical-Stability Note

The final system was tested on an NVIDIA Tesla T4.

During inference, the default pre-Ampere FP16 diffusion path produced invalid/NaN latent values in the tested environment.

For the final project pipeline, the ACE-Step pre-Ampere CUDA fallback was therefore changed to:

torch.float32

for improved numerical stability.

This is an inference-environment workaround and is not part of the LoRA adapter weights.

---

Evaluation

A controlled ACE-Step Base vs ACE-Step + LoRA experiment was performed.

Experimental Setup

The evaluation used:

• 5 fixed prompt/lyric conditions
• 3 seeds per condition
• Base and LoRA model conditions
• identical generation settings
• 30 generated audio clips in total

The evaluation seeds were:

101, 202, 303

Each generated clip was 30 seconds long.

CLAP Evaluation

CLAP text-audio similarity was used as an automated semantic-alignment metric.

Each 30-second clip was evaluated over three segments:

0–10 seconds
10–20 seconds
20–30 seconds

The three segment scores were averaged to obtain the final score for each clip.

Results

Statistical Tests

Paired t-test:

t = -2.7971
p = 0.0143

Wilcoxon signed-rank test:

W = 17.0
p = 0.0125

---

Evaluation Interpretation

The LoRA did not improve general CLAP text-audio semantic similarity in this experiment.

This does not mean that the ACE-Step base model is necessarily better in overall musical quality.

CLAP does not directly measure:

• Nepali pronunciation
• exact lyric adherence
• vocal naturalness
• perceived music quality
• traditional instrumentation quality
• cultural authenticity
• listener preference

The CLAP result should therefore be interpreted as one automated evaluation signal rather than a complete assessment of the generated music.

The significance tests should also be treated as exploratory because the 15 paired observations were produced from five prompt conditions with three seeds each rather than 15 completely independent prompts.

---

Important Training Limitation

The training dataset primarily represented Nepali folk musical style and instrumentation.

It was not a large, precisely aligned dataset containing:

Nepali written lyrics
        ↕
exact sung pronunciation
        ↕
matching vocal audio

Therefore, this adapter should primarily be understood as a:

Nepali folk style/domain adaptation

rather than a dedicated Nepali singing-pronunciation model.

It can influence musical style and instrumentation without guaranteeing exact pronunciation or realization of every supplied lyric.

---

Download the Adapter

from huggingface_hub import snapshot_download


LORA_REPO = (
    "pradipbasnet68/"
    "nepali-folk-acestep-1.5-lora"
)


lora_path = snapshot_download(
    repo_id=LORA_REPO
)


print(lora_path)

---

Load the LoRA

After initializing ACE-Step 1.5:

handler.load_lora(
    lora_path
)

handler.set_use_lora(
    True
)

handler.set_lora_scale(
    0.50
)

print(
    "Nepali Folk LoRA enabled."
)

The ACE-Step base model and required checkpoints must be loaded separately.

---

Example Music Prompt

An example folk-oriented music description used with the project pipeline is:

authentic traditional Nepali lok geet,
nostalgic emotional Nepali folk song,
warm expressive Nepali singing,
traditional wooden madal,
expressive sarangi,
bamboo bansuri,
acoustic village folk performance,
natural traditional Nepali folk arrangement,
clear Nepali vocals,
no synthesizer,
no electronic drums,
no electric guitar

Nepali lyrics are supplied separately through ACE-Step's lyric-conditioning input.

---

Related Project Model

The final lyric-generation model is:

pradipbasnet68/gemma-3-4b-nepali-lyrics-lora

It generates Nepali Devanagari lyrics before the ACE-Step generation stage.

---

Limitations

Generated outputs may contain:

• inaccurate Nepali pronunciation
• missing or altered lyric phrases
• unnatural vocals
• repeated lyrics
• inconsistent musical structure
• unexpected instrumentation
• variation between random seeds

Generation quality depends on the supplied lyrics, prompt, LoRA scale, seed, audio duration, and ACE-Step inference settings.

This adapter does not guarantee accurate Nepali vocal synthesis.

---

Responsible Use

This adapter was developed as an experimental academic research model.

Generated music should be reviewed manually before publication or distribution.

The model should not be assumed to represent every Nepali regional style, dialect, musical community, or folk tradition.

Users are responsible for complying with the ACE-Step license and any applicable rights associated with prompts, lyrics, datasets, and generated audio.

---

Base Model

This adapter requires:

ACE-Step/Ace-Step1.5

The base weights are not included in this repository.

The final project used the ACE-Step 1.5 Turbo checkpoint for inference.

---

Author

Pradip Raj Basnet

Developed as part of an undergraduate final-year project investigating AI-based Nepali folk music generation.

---

Disclaimer

This repository contains an experimental LoRA adapter developed for academic research.

Performance may vary depending on ACE-Step version, hardware, prompts, lyrics, random seeds, and inference configuration.