lky-qlora

lky-qlora: a Lee Kuan Yew reasoning-style LoRA for Qwen3-14B

A LoRA adapter that gives Qwen3-14B the reasoning style of Lee Kuan Yew. It learns four habits from his public speeches, interviews, and press conferences: reframe the question before answering, argue from first principles and national interest, use concrete historical analogies, and mark what it cannot be sure of. Source material comes from the National Archives of Singapore.

Disclaimer. This is a research and education project. The outputs are AI-generated imitations of a reasoning style. They are not real statements by Lee Kuan Yew and may be inaccurate. This work is not affiliated with or endorsed by his estate or the Government of Singapore.

Why this exists

The goal is to test how far a small, locally-runnable open-source model can be pushed with cheap, lightweight training. Instead of calling a frontier API, can a 14B model that runs on one 16GB consumer GPU be given the reasoning style of a specific expert? That means the way they frame a problem, the priors they argue from, and how they handle uncertainty.

Lee Kuan Yew is a good test case. His reasoning method is distinctive and well-documented, and there is a large public record to learn from. The result is a roughly 1GB adapter, trained overnight on one consumer GPU for a few dollars of API cost, that measurably shifts the base model toward that method (see Evaluation below). The pipeline is a reusable template. Swap the source material and you can teach any well-documented expert's reasoning to a model you own and run offline. It is fully open source.

Intended use

Research into style transfer and historical-persona modelling, and educational exploration of a reasoning style. Not for impersonation, and not for presenting outputs as authentic statements by the real person.

How it was made

• Base: Qwen3-14B, non-thinking mode. His reasoning should be the visible answer, not a hidden scratchpad.
• Method: QLoRA, 4-bit, rank 64, alpha 128, on all linear layers. Loss is applied only to his turns.
• Data: about 4.0M tokens. Interview and press-conference exchanges as multi-turn chats, speeches paired with generated questions, and about 10% generic instruction data to keep general instruction-following. Each example has a dated system prompt so the era is selectable at inference.
• Dataset: sjsim/lky-reasoning-recipe
• Pipeline and full method: https://github.com/pixiiidust/lky-brain

Evaluation

Judged by claude-opus-4-8 on 24 held-out interview questions that none of the models trained on. Six reasoning moves, plus a 1 to 5 voice score:

This is the epoch-2 checkpoint. A fully-trained epoch-3 adapter reached a lower training loss but overfit. On unseen questions it lost half its "bounds its uncertainty" and its question-reframing, because it had memorised his confident, declarative register. Epoch-2 generalises better and ties epoch-3 on voice. Note the eval is n=24, so the smaller per-move numbers are directional.

Usage

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
import torch

tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-14B")
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3-14B",
    quantization_config=BitsAndBytesConfig(load_in_4bit=True,
        bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16),
    device_map={"": 0})
model = PeftModel.from_pretrained(base, "sjsim/lky-qlora")

msgs = [
    {"role": "system", "content": "You are Lee Kuan Yew, Prime Minister of "
     "Singapore, speaking candidly in an interview. It is August 1965."},
    {"role": "user", "content": "Will Singapore survive as an independent nation?"},
]
prompt = tok.apply_chat_template(msgs, tokenize=False,
    add_generation_prompt=True, enable_thinking=False)
ids = tok(prompt, return_tensors="pt").to("cuda")
out = model.generate(**ids, max_new_tokens=500, do_sample=True,
    temperature=0.8, top_p=0.9)
print(tok.decode(out[0][ids["input_ids"].shape[1]:], skip_special_tokens=True))

Keep enable_thinking=False. The style lives in the visible answer.

Limitations

• It imitates a style. It is not a source of his real views and will confidently produce statements he never made.
• The source is mostly English. Some records are translations of Hokkien, Malay, or Mandarin remarks, so the phrasing there is a translator's, not his.
• It inherits Qwen3-14B's knowledge cutoff and biases, and can be anachronistic.
• The eval is n=24. Treat the smaller per-move numbers as directional.