ft-llm-storytelling-lora

ft-llm-storytelling-lora

LoRA adapters for Qwen 2.5 7B Instruct, fine-tuned to generate immersive YouTube storytelling scripts in French and English (style EGO, Feldup, Lemmino, Squeezie…).

Learning project — exploring LLM fine-tuning via QLoRA and Unsloth end-to-end.
Full pipeline and code: github.com/JordyBacherot/FineTunning_LLM_StoryTelling

---

Model Details

• Developed by: Jordy
• Model type: LoRA adapter (PEFT) — requires merging with base model before serving
• Base model: unsloth/Qwen2.5-7B-Instruct-bnb-4bit
• Languages: French (54 %), English (45 %)
• Fine-tuned for: YouTube narrative script generation (storytelling style)
• Framework: PEFT 0.18.1 / Unsloth / TRL SFTTrainer

---

Training Data

273 examples built from transcripts of 20 YouTube channels known for storytelling quality:

• FR (11 channels): ego_one, feldup, squeezie, whatafail, thegreatreview, kombo000, sylvqin, ggmilgram, micode, alt236, mamytwink
• EN (9 channels): lemmino, nexpo, barelysociable, summoningsalt, internethistorian, coffeezilla, bobbybroccoli, wendigoon, fredrikknudsen

Pipeline: yt-dlp extraction → Ollama cleaning (Scribe Fidèle, style-preserving) → Groq reverse prompting → ChatML format with 3 persona levels (base / archetype / youtuber-specific).

---

Training Hyperparameters

Best checkpoint: epoch 3 — val_loss = 2.3515

---

Evaluation

Judge: Llama 3.3 70B via Groq — 5 criteria: accroche, tension narrative, rétention, naturel du style, fidélité style.

The vanilla baseline scores slightly higher on the automated judge, which tends to favor generic fluent prose. The fine-tuned model has acquired YouTube storytelling characteristics (energy, narrative tension, direct address) that the LLM judge partially penalizes.

Known limitations:

• Occasional repetition loops (mitigated by repetition_penalty: 1.3)
• Structural inconsistencies on some outputs — symptom of a small dataset (273 examples)
• Style specificity limited to channels with dedicated personas (notably ego_one)
• May hallucinate plausible but incorrect facts

---

How to Use

These are LoRA adapters — not a standalone model. You must merge them with the base model before serving with vLLM.

Merge with base model (~20 min, ~15 GB RAM)

import torch, gc
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B-Instruct", dtype=torch.bfloat16, device_map="cpu"
)
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

model = PeftModel.from_pretrained(base, "JordyBach/ft-llm-storytelling-lora")
model = model.merge_and_unload()
gc.collect()

model.save_pretrained("merged_model", max_shard_size="2GB")
tokenizer.save_pretrained("merged_model")

Prompt format (ChatML — must match training distribution)

system_prompt = "Tu es un narrateur-storyteller YouTube..."  # or channel-specific persona

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": "[LANG:FR]\nTon sujet ici"},
]

The [LANG:FR] / [LANG:EN] prefix in the user message is critical — it was present in all training examples.

Serve with vLLM (after merging)

python -m vllm.entrypoints.openai.api_server \
  --model merged_model \
  --quantization bitsandbytes \
  --load-format bitsandbytes \
  --max-model-len 16000 \
  --enforce-eager

Recommended inference params: repetition_penalty=1.3, stop=["---", "\n#", "**Note"]

---

Bias, Risks, and Limitations

• Training data is sourced from public YouTube transcripts — content reflects the style and potential biases of the selected creators
• The model may generate factually incorrect content (hallucinations)
• Not suitable for factual question answering or tasks requiring accuracy
• Output quality degrades significantly outside the YouTube storytelling domain