linlu-lora-v0.3-qwen3.5-9b

Lin Lu (林路) persona LoRA — v0.3

Production adapter for the 林路 character with timed multi-bubble replies and sticker catalog refs ([[sticker:挠头]] etc.), trained from the Divergence 2% production corpus (204 user-days, 8,559 assistant turns).

vs v0.2: preserves [[sticker:ref]] markers (not flattened to [发送了一张表情]), prefixes every bubble with [HH:MM] timestamps, and uses the full 204-day production dataset.

Recommended successor: linlu-lora-v0.4-qwen3.6-27b — the 27B base with this same data pipeline; better depth, sticker usage, and pragmatics (eval loss 1.769 vs 1.869). This 9B remains useful when you only have ~20 GB VRAM.

---

Requirements

pip install "transformers>=5.5" peft torch accelerate fastapi uvicorn

You need a GPU with ~20 GB VRAM for bf16 inference (9B base + LoRA). trust_remote_code=True is required for Qwen3.5.

---

Quick start (Python)

import re
import torch
import transformers
from peft import PeftModel
from transformers import AutoTokenizer, AutoConfig

BASE = "Qwen/Qwen3.5-9B"
ADAPTER = "s-g-labs/linlu-lora-v0.3-qwen3.5-9b"

arch = AutoConfig.from_pretrained(BASE, trust_remote_code=True).architectures[0]
cls = getattr(transformers, arch)
tok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)
model = cls.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto",
                            trust_remote_code=True)
model = PeftModel.from_pretrained(model, ADAPTER)
model.eval()

def chatml(messages: list[dict]) -> str:
    s = ""
    for m in messages:
        s += f"<|im_start|>{m['role']}\n{m['content']}<|im_end|>\n"
    return s + "<|im_start|>assistant\n"

system = """你将扮演「林路」，一个真实、有血有肉的角色，正在用中文和「对方」聊天。
每条消息单独一行，以发送时刻开头，格式为「[HH:MM] 消息内容」。
发表情包时整条消息写作「[HH:MM] [[sticker:表情名]]」。"""

messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": "[19:02] 林老师晚上好呀"},
]
ids = tok(chatml(messages), return_tensors="pt").to(model.device)
eos = [tok.eos_token_id, tok.convert_tokens_to_ids("<|im_end|>")]

with torch.no_grad():
    out = model.generate(
        **ids, max_new_tokens=512, do_sample=True, temperature=0.7,
        top_p=0.95, top_k=20,
        # Do NOT set no_repeat_ngram_size — timed format repeats "[HH:MM] " prefixes.
        eos_token_id=eos, pad_token_id=tok.pad_token_id or eos[0],
    )
raw = tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True)
print(raw)

---

OpenAI-compatible server (included)

This repo ships a tested FastAPI server under serve/:

git clone https://huggingface.co/s-g-labs/linlu-lora-v0.3-qwen3.5-9b
cd linlu-lora-v0.3-qwen3.5-9b/serve
pip install -r requirements.txt

# Downloads base from HF on first run if not cached locally
CUDA_VISIBLE_DEVICES=0 python server_openai.py \
  --base Qwen/Qwen3.5-9B \
  --adapter .. \
  --port 8000

Endpoints (matches wl2.studio Go proxy):

---

Output format

<think>
{Chinese inner monologue — optional}
</think>
[19:05] 晚上好呀
[19:06] [[sticker:挠头]]
[19:06] 在忙吗？

Frontend: load /api/stickers (or a static catalog) and replace [[sticker:ref]] with <img src="...">. See serve/sticker_refs.js for reference client code.

---

Recommended sampling

---

Training config

See training_config.yaml. Data prep script: prepare_data_v3.py (not bundled; contact repo maintainers). ShareGPT JSONL with timestamped bubbles and sticker refs.