pickbits-voice-daily-v1

pickbits-voice-daily-v1

A LoRA adapter that writes in the house voice of PickBits Daily Signal, trained on ~4 months of shipped, human-approved editorial content. Style-transfer proof of concept: the adapter reliably reproduces the publication's register, structure, and cadence on unseen topics, where the base model produces generic (often wrong-genre) text.

• Base model: Qwen/Qwen2.5-1.5B-Instruct (frozen; this repo contains only the adapter)
• Method: plain bf16 LoRA — r=16, alpha=32, dropout 0.05, targeting q/k/v/o + gate/up/down projections (18.46M trainable params, 1.18% of base)
• Data: 815 chat-format pairs harvested from shipped content (94 daily editions: story sections, tl;dr bullets, standfirsts; plus devlog social captions). 40 pairs from the newest publication dates held out — zero date leakage.
• Training: 3 epochs / 153 steps, lr 1e-4 cosine, effective batch 16, max_len 1024, assistant-only loss masking. 19.5 min on a single RTX 3080 Ti (12 GB). Loss 2.98 → 2.39.

Before / after (held-out prompt, never seen in training)

Prompt: Engine: daily-signal / Task: story section / Heading: "Google put a deepfake button on the whole planet — then yanked it inside a day." / Seed: "The tool didn't invent fakery…"

Base Qwen2.5-1.5B-Instruct:

In an era where truth was as elusive as a shadow cast by a passing cloud, humanity found itself at the mercy of an unprecedented technology…

(Dystopian fiction — wrong genre entirely.)

With this adapter:

The tool didn't invent fakery; it let a lie inherit the credibility of the real map underneath it. On Monday, August 1, Google's…

(Correct genre, register, and house structure: seed-line opener, declarative key-fact cadence.)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen2.5-1.5B-Instruct"
tok = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base, dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, "MrPIck/pickbits-voice-daily-v1")

messages = [
    {"role": "system", "content": "You are the PickBits staff writer. Write in the house voice."},
    {"role": "user", "content": "Engine: daily-signal\nTask: tldr bullet\nSeed: EU AI Act enforcement began today with..."},
]
ids = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=False).to(model.device)
out = model.generate(ids, max_new_tokens=400, temperature=0.7, top_p=0.9, do_sample=True)
print(tok.decode(out[0, ids.shape[1]:], skip_special_tokens=True))

Limitations

• Style, not knowledge. Prompts in training carried only a heading and seed sentence, so the adapter learned voice and structure — it will confidently invent facts. Feed real source facts in the prompt; let the adapter supply only the how, not the what.
• Trained for specific task framings (Engine: daily-signal, Task: story section | tldr bullet | edition standfirst); other prompt shapes are out of distribution.
• English only; single narrow editorial voice by design.

Provenance

Training data is proprietary shipped editorial content from PickBits (not included in this repo). Adapter weights only. Built as a one-off proof of concept for a harvest → pairs → LoRA → held-out-eval pipeline, 2026-08-09.