valleygirl-1.5b

valleygirl-1.5b

GitHub repo with all code here.

A LoRA fine-tune of Qwen/Qwen2.5-1.5B-Instruct, merged into full weights, that answers questions in an exaggerated "valley girl" persona: it briefly addresses whatever you asked, then steers the conversation toward personal drama (relationships, who-said-what, projection onto the user), and doubles down on that redirection even when pushed back on.

Try it live: benlahner/valleygirl (Gradio Space on ZeroGPU).

Model Details

• Base model: Qwen/Qwen2.5-1.5B-Instruct
• Method: LoRA fine-tuning via TRL's SFTTrainer, then merged into the base weights with merge_and_unload() and pushed as a standalone model (not an adapter).
• Model type: Causal decoder-only LLM, text-generation
• License: apache-2.0 (inherited from the base model)

Uses

Direct Use

Casual/entertainment chatbot with a consistent comedic persona. Not intended for factual Q&A — it deliberately deflects direct questions.

Out-of-Scope Use

Not suitable for tasks requiring straightforward, on-topic answers, factual reliability, or professional/production use cases. Not evaluated for safety-critical or high-stakes deployments.

Bias, Risks, and Limitations

The persona is trained to redirect conversations toward interpersonal topics regardless of the user's actual question, which is intentional but means the model will not reliably follow instructions or answer directly. Training data was synthetically generated (see below) and has not been audited for bias beyond the intended persona.

How to Get Started with the Model

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_REPO = "benlahner/valleygirl-1.5b"

tokenizer = AutoTokenizer.from_pretrained(MODEL_REPO)
model = AutoModelForCausalLM.from_pretrained(MODEL_REPO, dtype=torch.bfloat16).to("cuda")
model.eval()

messages = [{"role": "user", "content": "How does photosynthesis work?"}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt", return_dict=True
).to("cuda")

with torch.no_grad():
    output = model.generate(
        **inputs, max_new_tokens=200, do_sample=True,
        temperature=0.7, pad_token_id=tokenizer.eos_token_id,
    )

print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Training Details

Training Data

A synthetically generated, multi-turn SFT dataset (451 train / 47 eval examples) built from a fixed set of seed questions spanning science, history, math, technology, and culture. Each example pairs a straightforward question with a valley-girl-voiced response that briefly acknowledges the question before pivoting to personal drama, generated to be consistent with the target persona described above.

Training Procedure

• Framework: TRL SFTTrainer with a PEFT LoRA adapter, later merged into the base model
• LoRA config: r=16, alpha=32, dropout=0.05, target modules q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, bias="none"
• Epochs: 3
• Batch size: 4 per device, gradient accumulation 4 (effective batch size 16)
• Learning rate: 1e-4, cosine schedule, warmup ratio 0.03
• Precision: bf16
• Max sequence length: 2048
• Eval/save strategy: per epoch

Compute Infrastructure

Single-GPU fine-tuning (fits comfortably on one consumer/workstation GPU given the 1.5B parameter count and LoRA).

Environmental Impact

Not measured. Given the small model size (1.5B params), LoRA training, and short training run (3 epochs over ~450 examples), compute footprint is minimal relative to full pretraining runs.