Downloads · 30 days
31
9% of all-time downloads
barath007183/qwen2.5-1.5b-yoda-speak
qwen2.5-1.5b-yoda-speak is a text generation model from barath007183. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
A LoRA fine-tune of Qwen2.5-1.5B-Instruct that translates ordinary English sentences into Yoda-style syntax (object/verb-first reordering, e.g. "Read this you must.").
Downloads · 30 days
31
9% of all-time downloads
All-time downloads
364
Public
Parameters
1.5B
3.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors3.1 GB · 100%
From the Hugging Face model README
A LoRA fine-tune of Qwen2.5-1.5B-Instruct that translates ordinary English sentences into Yoda-style syntax (object/verb-first reordering, e.g. "Read this you must.").
This model takes a plain English sentence and rewrites it in Yoda's speaking style — primarily through object-subject-verb reordering rather than vocabulary changes. It was trained as a hands-on learning project to understand LoRA fine-tuning mechanics end-to-end: data preparation, chat-template formatting, loss masking, hyperparameter tradeoffs, and — most importantly — why validation loss alone isn't sufficient for picking a checkpoint on a small dataset.
Prompt the model with an instruction to translate a sentence into Yoda-speak. Works best on single, self-contained declarative sentences, questions, negations, and imperatives similar in length to the training data (roughly 25-60 characters). Style transfer only — no factual knowledge was added.
dvgodoy/yoda_sentences dataset — a narrow, templated dataset of short declarative sentences about mundane objects. Generalization to genuinely novel sentence structures (questions, multi-clause sentences, dialogue) is measurably weaker than in-distribution performance.For inputs meaningfully different from short declarative English sentences (long paragraphs, dialogue, technical text), manually review output quality before relying on it.
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
model_name = "barath007183/qwen2.5-1.5b-yoda-speak" # replace with actual repo id
model = AutoModelForCausalLM.from_pretrained(model_name, dtype=torch.bfloat16, device_map="cuda")
tokenizer = AutoTokenizer.from_pretrained(model_name)
messages = [
{"role": "system", "content": "You are a Yoda-speak translator."},
{"role": "user", "content": "Translate this into Yoda-speak: The cat sat on the mat."},
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
output = model.generate(**inputs, max_new_tokens=60, do_sample=False)
print(tokenizer.decode(output[0][inputs['input_ids'].shape[1]:], skip_special_tokens=True))
# "Sat on the mat, the cat did."
dvgodoy/yoda_sentences — 720 paired examples of plain English sentences and their Yoda-syntax translations. Split 80/10/10 (576 train / 72 validation / 72 test), stratified by random shuffle (seed 42).
Each row was formatted as a 3-turn chat example (system + user + assistant) using Qwen2.5's native ChatML template via tokenizer.apply_chat_template(). A fixed system prompt ("You are a Yoda-speak translator.") was paired with one of 4 randomly-rotated user instruction phrasings per row, to avoid the model overfitting to a single trigger phrase rather than the underlying task.
q_proj, k_proj, v_proj, o_proj (attention only — MLP modules were not targeted, since this task is primarily syntactic/relational rather than knowledge-based)assistant_only_loss=True (trl 1.8.0) — loss computed only on assistant-response tokens, not system/user tokensEvaluation was done in three stages, deliberately going beyond validation loss alone:
Three checkpoints (epoch 2, 3, and 8) were compared at each stage, alongside the un-fine-tuned base model as a control.
Qwen2.5-1.5B-Instruct architecture (transformer decoder, SwiGLU MLP blocks), causal language modeling objective, fine-tuned via supervised fine-tuning (SFT) with LoRA adapters merged into the base weights post-training.
1x NVIDIA RTX 4090 (24GB VRAM), rented via RunPod (PyTorch template, CUDA 12.8 driver)
transformerstrl 1.8.0peft 0.19.1torch 2.x (cu124 build)datasetsBarath C