Downloads · 30 days
12
11% of all-time downloads
AlekseyCalvin/Lyrical_MT_ru2en_var1_Qwen25_3B_ORPO_r128ft
Lyrical_MT_ru2en_var1_Qwen25_3B_ORPO_r128ft is a translation model from AlekseyCalvin. Use it when you need text moved from one language to another. The card lists the license as apache-2.0.
By Silver Age Poets <br Variant 0.1: ORPO-tuned Qwen2.5-3B Model Weights
Downloads · 30 days
12
11% of all-time downloads
All-time downloads
107
Public
Parameters
3.1B
6.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors6.2 GB · 100%
From the Hugging Face model README
By Silver Age Poets <br> Variant 0.1: ORPO-tuned Qwen2.5-3B Model Weights
Fine-tuned leveraging Odds Ratio Preference Optimization (ORPO) on our Russian-to-English song lyrics translation/localization dataset. <br> Our aim with this work is to foster a translation model capable of adaptively localizing idiomatic, formal/poetic/rhythmic, and performance-catered features of lyrical input texts, whilst retaining adequate accuracy at the level of direct semantic translation. <br> The training was held for 6 Epochs at Rank 128. <br>
This is an experimental/WIP fine-tune, and probably first of many. We find that its zero-shot translations (without extensive specialized instruction templating/priming) are already generally more reminiscent of input songs' phrasal rhythm/meter and expressive tone, as compared to base model outputs. <br> However, it makes far more semantic errors/mis-attributions and/or arbitrary substitutions (straying from translation into extrapolative composition). <br> We would appreciate feedback/reports from anyone else who happens to try out this model, or its other variants (to be released in the near future). <br>
Next, we will attempt to fine-tune a larger or/and an already translation-specialized model, and continue experimenting with hyperparameters/techniques. <br>