Qwen3-4B-Base-ThinkCode-A-U025

Qwen3-4B-Base-ThinkCode-A-U025 — PEFT Adapter

This repository contains a PEFT LoRA adapter only. It does not contain the Qwen3 base-model weights and cannot be loaded as a standalone causal language model.

The required base is Qwen/Qwen3-4B-Base at the fixed revision 906bfd4b4dc7f14ee4320094d8b41684abff8539.

Adapter construction

A-U025 is the Phase A uniform-scale arm. Starting from the completed source LoRA, every selected LoRA B tensor—including the lm_head adapter—is multiplied by 0.25 in FP32. LoRA A tensors are unchanged. With lora_alpha=128 and r=64, PEFT therefore applies the exact intended 0.25× source delta to all 253 adapted modules.

Because Qwen3 ties lm_head.weight to embed_tokens.weight, the released standard-PEFT representation stores the head factors as transposed embed_tokens LoRA factors and sets ensure_weight_tying=true. PEFT then shares that adapter with the tied output layer, matching both input-embedding and output-head effects without storing any base-layer tensor.

The repository includes MODULE_SCALE_MANIFEST.json, which records every logical module, tensor key, physical base weight, and scale. This release is from the completed Phase A delta-scaling line; it is not the later failed NEXTGEN route and does not include subsequent protocol-repair experiments.

Loading with PEFT

Use recent transformers and peft versions. Load the fixed base first, then attach this adapter:

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3-4B-Base"
base_revision = "906bfd4b4dc7f14ee4320094d8b41684abff8539"
adapter_id = "modrill/Qwen3-4B-Base-ThinkCode-A-U025"

tokenizer = AutoTokenizer.from_pretrained(base_id, revision=base_revision)
base = AutoModelForCausalLM.from_pretrained(
    base_id,
    revision=base_revision,
    torch_dtype="auto",
    device_map="auto",
)
model = PeftModel.from_pretrained(base, adapter_id)

messages = [{"role": "user", "content": "Write a Python function that checks whether a number is prime."}]
prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
eos_ids = [
    tokenizer.eos_token_id,
    tokenizer.convert_tokens_to_ids("<|im_end|>"),
]
outputs = model.generate(**inputs, max_new_tokens=2048, eos_token_id=eos_ids)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[-1]:], skip_special_tokens=True))

The base tokenizer's chat template supports enable_thinking. Disable it for direct code generation matching the concise screening style, or enable it when explicit reasoning is desired. Pass both <|endoftext|> and <|im_end|> as EOS IDs. Keep the combined prompt and generated sequence within 32K tokens, the fixed base model configuration limit, unless a separate long-context extension is validated.

Development evaluation

On the corrected EvalScope Full1055 development suite, the preregistered seed=3407 code_only result was 266/1055 = 25.21%. Independent forward and reverse scoring produced 0 verdict flips.

This is a single-seed development screening result, not formal confirmation, a held-out estimate, or a multi-seed aggregate. No aggregate from A-NH025 is attributed to this adapter.

Limitations

• This adapter requires the exact base model and should not be loaded alone.
• The published evidence is development-only and single-seed.
• Generated code can be incorrect, insecure, or non-compiling; sandbox and test it independently.
• No production safety, security, or suitability certification is implied.

License

The fixed base card and included license identify Apache-2.0. This adapter preserves that license text and metadata. Users should independently verify the upstream Qwen3 license, notices, training-data terms, and applicability to their use case.