rogan-27b

rogan-27b

A LoRA adapter for Qwen/Qwen3.8-27B that answers questions in the register of long-form podcast conversation — spoken cadence, digressions, and all — rather than in standard assistant prose.

Trained with Tinker using the Thinker UI.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.8-27B", device_map="auto")
model = PeftModel.from_pretrained(base, "lalopenguin/rogan-27b")
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.8-27B")

Apply the non-thinking chat template (qwen3_8_disable_thinking). The adapter was trained with thinking disabled; sampling it with the default reasoning template produces malformed output — visible </think> fragments and leaked reasoning text.

Training

Data construction

Answers are verbatim spans of podcast transcript. Questions are synthetic: a local instruct model read each span and wrote a question it answers. Only the question side is model-generated — the answer side is left untouched, so the adapter learns the source register rather than a teacher model's voice.

Two filters proved necessary. An earlier version trained without them produced a model that parroted the prompt back mid-answer and began replies mid-sentence:

• Sentence snapping. Transcript chunks are fixed-size slices that open and close mid-thought. Partial sentences at both ends are trimmed. Answers starting at a sentence boundary: 10% → 97%.
• Echo filtering. Interview subjects often say the question aloud, so a question written about a span frequently appears inside it. Pairs where more than half the question's topic words recur in the answer are dropped. Mean overlap 42% → 29%; pairs above 50%: 29% → 0%.

Limitations

• Single plain questions work best. Over 99% of training prompts were one direct question. Multi-part or instruction-shaped prompts ("do X and include Y") are out of distribution — the model tends to restate the instruction rather than follow it.
• Replies often open mid-conversation — "Yeah", "That's true", "And what you said…" — because 38% of training answers begin on a conversational turn. It may agree with something you never said.
• Not a factual source. This adapter models how a conversation sounds, not whether its claims are true. It reproduces the opinions and errors of its source material and should not be relied on for factual questions.
• No safety tuning beyond whatever the base model carries.

Running locally on Apple Silicon (MLX)

Two things bite when merging this adapter. Both are recorded here because neither is obvious from the files.

1. The linear-attention projection is fused in every published checkpoint. Tinker trains in_proj_q, in_proj_k and in_proj_v as three separate LoRA pairs, but Qwen/Qwen3.8-27B — and every MLX/GGUF build derived from it — stores one fused in_proj_qkv. There is no 1:1 key mapping, and downloading the bf16 base does not help; it is fused there too. Stack the three deltas along the output axis in [q, k, v] order, matching how the gated-delta-net splits them back out (mx.split at [key_dim, 2*key_dim]).

Key mapping into an MLX checkpoint:

Delta scaling is lora_alpha / r = 1.0. Merging straight into a 4-bit MLX base (dequantize -> add -> requantize, group_size 64, bits 4, affine) peaks around 15.5 GB of RAM and needs no bf16 copy.

2. mlx_lm overrides the thinking setting. TokenizerWrapper.apply_chat_template injects enable_thinking=True for any model whose template mentions thinking, so editing the template's default has no effect and the model emits leaked reasoning ("We need answer user's question: ..."). Pin the branches instead - {%- if false %} on the reasoning-instructions block and {%- if true %} on the empty-think-block branch - so the variable is ignored. mlx_lm chat has no --chat-template-config flag, so pinning is the only fix that works for interactive use.

With both done, it runs at roughly 6.5 tok/s on an M4 Mac mini at ~15.4 GB peak.

Ollama is not an option: it removed LoRA adapter support.

License

The base model is Apache 2.0. Training data derives from third-party podcast transcripts; the transcripts themselves are not redistributed here.