qwen2.5-1.5b-riddle-lora

Qwen2.5-1.5B-Instruct LoRA for Riddle Solving

LoRA adapter fine-tuned on a mixed dataset of ~1,200 riddles, evaluated on 66 held-out riddles.

Held-out exact match: 63.6% (42/66) — beats the from-scratch diffusion baseline (47.0%) by +16.6 pp.

Quick start

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

model_name = "Qwen/Qwen2.5-1.5B-Instruct"
adapter = "beme08/qwen2.5-1.5b-riddle-lora"

tokenizer = AutoTokenizer.from_pretrained(model_name)
if tokenizer.pad_token is None:
    tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForCausalLM.from_pretrained(
    model_name, torch_dtype=torch.float32, trust_remote_code=True
).to("cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu")

model = PeftModel.from_pretrained(model, adapter)

def solve(riddle):
    messages = [
        {"role": "system", "content": "Solve the riddle. Respond with only the answer and no explanation."},
        {"role": "user", "content": riddle},
    ]
    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    out = model.generate(**inputs, max_new_tokens=60, do_sample=False, pad_token_id=tokenizer.eos_token_id)
    return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True).strip()

print(solve("What gets wetter the more it dries?"))
# → "A towel"

Evaluation

66 held-out riddles (same split hash 5b05f514e666c2dd as the diffusion study). Exact match uses canonicalized answers (lowercased, punctuation stripped, deduplicated tokens).

Multi-seed mean (seeds 42, 123, 456): 60.1% ± 3.8 pp (SD), range 56.1–63.6%.

Training details

• Base model: Qwen/Qwen2.5-1.5B-Instruct
• Adapter: LoRA r=8, α=16, dropout=0.1 on q_proj, k_proj, v_proj, o_proj
• Data: 177 original riddles (GPT-generated for the diffusion study) + ~1,000 riddles from RiddleSense (MIT), Kaggle (CC0), and Crawsome (Unlicense)
• Training: 10 epochs, batch size 4, AdamW (lr=2e-4), cosine schedule, early stopping by validation EM
• Hardware: Apple M4 Pro (MPS), float32
• Seed: 42

External riddle answers were normalized to match the diffusion convention: lowercase, articles added for single-word common nouns.

CLI

Install the diffusion CLI with optional LoRA backend:

pip install "riddle-diffusion[lora]"
riddle-solve --model diffusion "what gets wetter as it dries"
riddle-solve --model qwen-lora "what gets wetter as it dries"

The LoRA backend downloads the 1.5B base model on first use (~3 GB).

Links

• Full blog post
• Diffusion model (8M params)
• GitHub repository
• RIDDLE_DIFFUSION_REPORT.md