qwen3.8-27b-merged

qwen3.8-27b-merged

unsloth/Qwen3.8-27B with a LoRA fine-tune merged into the weights, saved in 16-bit. Ready to load or serve directly, with no adapter composition step.

The standalone adapter is at DanielTobi0/qwen3.8-27b-lora (about 435 MB) if you already have the base weights and would rather not pull ~55 GB.

[!IMPORTANT] This model was trained on prompts rendered with reasoning_effort="xhigh". The Qwen3.8 chat template injects a different instruction line (or none at all) for other effort levels, so serving it at another setting changes the system prefix it was conditioned on. Pass reasoning_effort="xhigh" when you apply the chat template — see below.

Loading

import torch
from transformers import AutoModelForImageTextToText, AutoTokenizer

REPO = "DanielTobi0/qwen3.8-27b-merged"

model = AutoModelForImageTextToText.from_pretrained(
    REPO,
    dtype = torch.bfloat16,
    device_map = "auto",
)
model.eval()

tokenizer = AutoTokenizer.from_pretrained(REPO)

This is a native vision-language model (Qwen3_5ForConditionalGeneration), so it loads through AutoModelForImageTextToText. Use AutoModelForCausalLM instead only if you want the text-only decoder (Qwen3_5ForCausalLM) and intend to drop the vision tower. The weights are ~55 GB in bf16, so plan for an 80 GB accelerator, or pass a quantization_config to fit something smaller.

Qwen3.8's gated-deltanet layers have a fast Triton path that transformers does not ship. Without it you will see The fast path is not available ... Falling back to torch implementation and noticeably slower inference. Install flash-linear-attention and causal-conv1d, or load through Unsloth, which bundles them.

Generating

messages = [
    {"role": "system", "content": "You are xxxx AI. Respond authoritatively, precisely, and formally using 'we' without contractions or em dashes."},
    {"role": "user",   "content": "Which layer of the sovereign intelligence stack is partnered rather than owned?"},
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize = False,
    add_generation_prompt = True,
    reasoning_effort = "xhigh",      # must match training
)

inputs = tokenizer(text, return_tensors = "pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens = 1024, do_sample = False)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens = True))

Reading the output

add_generation_prompt=True ends the prompt with <|im_start|>assistant\n<think>\n, so generation begins already inside the thinking block. The model emits its reasoning, closes it with </think>, then writes the answer. Split on the closing tag to separate them:

reasoning, _, answer = decoded.partition("</think>")

Set enable_thinking=False in apply_chat_template to suppress reasoning, but note the model was trained exclusively on thinking-enabled examples.

Serving

The merged weights are a standard qwen3_5 checkpoint, so any runtime with Qwen3.8 support can serve them — for example:

vllm serve DanielTobi0/qwen3.8-27b-merged --dtype bfloat16

Pass the reasoning effort through your client's chat-template arguments (in an OpenAI-compatible request, chat_template_kwargs: {"reasoning_effort": "xhigh"}) so the system prefix matches training. Serving configuration was not exercised during this fine-tune — verify the rendered prompt before relying on it.

Training

Adapted modules

Qwen3.8-27B interleaves two attention types, so the fine-tune covered both rather than the usual seven names:

• q_proj, k_proj, v_proj, o_proj — the 16 full gated-attention layers
• in_proj_qkv, in_proj_z, out_proj — the 48 gated-deltanet linear-attention layers
• gate_proj, up_proj, down_proj — the MLP in all 64 layers

Targeting only the standard four attention names would leave 48 of 64 layers unadapted.

Limitations

The training set is small (236 samples) and was trained for 3 epochs at 2e-4, which is an aggressive schedule for a 27B model at this data scale. The model reliably reproduces the target register and reasoning format, but it does not reliably reproduce the factual content of the training data — spot checks show it confabulating specifics that contradict its own training examples. Treat it as a style and format fine-tune, not a knowledge base, and ground factual claims externally.