llama3-storyweaver-lora

Perfect 👌 — here’s a professional, polished Hugging Face model card you can paste directly into your model’s README.md at 👉 https://huggingface.co/cr-28/llama3-storyweaver-lora

---

🪶 LLaMA3-StoryWeaver-LoRA

LLaMA3-StoryWeaver-LoRA is a fine-tuned version of Meta’s LLaMA 3 model adapted with LoRA (Low-Rank Adaptation) to specialize in creative text generation — including mythology-inspired narratives, fantasy short stories, and character-driven tales.

It can generate engaging, coherent, and imaginative short stories across diverse mythological and fictional themes.

---

✨ Model Overview

---

📚 Training Details

• Dataset: A curated dataset of mythology and short story samples formatted in instruction–response pairs (JSONL).

• Dataset Size: ~18,500 training samples / ~2,000 validation samples.

• Objective: Fine-tune the model to generate vivid short stories with structured flow and strong creative language.

• Training Setup:

  • Batch size: adaptive (GPU memory–safe)
  • Epochs: 1
  • Gradient checkpointing: enabled
  • Mixed precision: fp16
  • Optimizer: AdamW
  • Checkpointing: every 500 steps (resumable training)

---

🧠 Model Capabilities

• Generates imaginative short stories, myths, and folklore.

• Capable of maintaining contextual coherence and creative tone.

• Can adapt prompts such as:

  • “Write a story about the birth of a sun god in modern times.”
  • “Tell a short fantasy about a cursed forest that sings at midnight.”
  • “Create a myth about how dreams were first created.”

---

🚀 Example Usage

from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "cr-28/llama3-storyweaver-lora"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

prompt = "Tell me a short story about a dragon who lost its fire."
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.8, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

---

📈 Evaluation (Qualitative)

(Evaluated manually on 100 generated samples.)

---

⚙️ How to Use the LoRA Adapter

If you wish to merge this adapter with the base LLaMA-3 model:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = "meta-llama/Meta-Llama-3-8B"
adapter_model = "cr-28/llama3-storyweaver-lora"

tokenizer = AutoTokenizer.from_pretrained(base_model)
model = AutoModelForCausalLM.from_pretrained(base_model)
model = PeftModel.from_pretrained(model, adapter_model)

---

⚠️ Limitations

• May hallucinate details or myths not based on factual sources.
• Occasionally overuses poetic or dramatic tone.
• Trained primarily on English, so other languages may produce inconsistent results.
• Not suitable for factual or scientific text generation.

---

🧩 Intended Use

• Creative writing assistants
• Story generation for games or prompts
• Mythology-based storytelling apps
• Inspiration for writers or educators

---

💡 Future Work

• Expand dataset to include world mythologies (Greek, Norse, Indian, Egyptian, etc.)
• Add multi-genre story training (sci-fi, mystery, historical)
• Incorporate reinforcement learning for narrative coherence (RLHF)

---

👨‍💻 Author

Model fine-tuned and prepared by: @cr-28

---

📜 License

This model follows the same licensing terms as LLaMA 3. Please ensure compliance with Meta’s LLaMA 3 License before commercial use.

---