qwen3-1.7b-shakespeare-lora

🎭 Qwen3-1.7B — Shakespearean LoRA

A LoRA fine-tuned version of Qwen/Qwen3-1.7B,
trained to respond in Shakespearean diction while retaining the reasoning ability of the base model.

🧠 Model Description

This model blends reasoning intelligence with Elizabethan poetic style.
It was fine-tuned using PEFT LoRA for efficient adaptation on a small GPU (T4),
using a curated dataset of modern English ↔ Shakespearean-style pairs.

• Base Model: Qwen/Qwen3-1.7B
• Fine-tuning Type: LoRA (low-rank adapter)
• Quantization: 4-bit NF4
• Objective: Imitate Shakespearean diction and sentence structure
• Language: English (Early Modern / Shakespearean)
• Frameworks: transformers, peft, bitsandbytes, accelerate
• Fine-tuned by: Sachinkry

⚙️ Model Sources

• Repository: Sachinkry/qwen3-1.7b-shakespeare-lora
• Base Model: Qwen/Qwen3-1.7B
• Dataset: Shakespearean-diction-dialogue

🪶 How to Use

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

base_model = "Qwen/Qwen3-1.7B"
adapter_model = "Sachinkry/qwen3-1.7b-shakespeare-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model, device_map="auto", load_in_4bit=True)
model = PeftModel.from_pretrained(model, adapter_model)

prompt = "Tell me about the beauty of the stars."
messages = [{"role": "user", "content": prompt}]
formatted = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(formatted, return_tensors="pt").to(model.device)

with torch.inference_mode():
    outputs = model.generate(**inputs, max_new_tokens=150, temperature=0.8)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Output Example:

The stars, those gleaming fires upon the firmament’s breast, whisper secrets of love and fate to the dreamers below."

📊 Training Details

Setting Value Base model Qwen/Qwen3-1.7B Dataset Sachinkry/shakespearean-diction-dialogue

Data size	1,040 examples (train/test split 80/20)
Epochs	3
Batch size (effective)	16 (accumulated)
Precision	FP16 mixed
Optimizer	paged_adamw_8bit
Learning rate	2e-4
Warmup ratio	0.05
Gradient checkpointing	Enabled
Quantization	4-bit NF4

Training Environment:

GPU: NVIDIA T4 (Google Colab)
Time: ~50 minutes
Frameworks: transformers, peft, bitsandbytes, accelerate

🧪 Evaluation

The fine-tuned model was evaluated using: LLM-as-a-Judge (Gemini 2.5 Flash) scoring on Shakespearean diction (0–5 scale). Average style score: 4.6 / 5 Model responses exhibit high lexical and syntactic alignment with Elizabethan English while preserving semantic coherence.

Example prompt:

User: "What is love?"
Model: “Love, that sweet and tender passion, doth bind hearts as iron clasps. It is the joy of shared dreams, the ache of parted farewells, and the purest bliss in all creation’s wide expanse.”

✅ Intended Use

• Direct Use: Creative writing, stylized responses, education, literary paraphrasing, and playful chatbot use.
• Out-of-Scope Use: Factual or scientific explanations requiring precision.
• Legal, medical, or sensitive decision-making.

⚠️ Bias, Risks, and Limitations

• Model style may exaggerate archaic speech and produce verbose or metaphorical phrasing.
• Not suitable for factual reasoning or multi-hop logical inference.
• Dataset primarily focuses on stylistic imitation, not truthfulness.