qwen3.8-27b-json-lora

Qwen3.8-27B Generic JSON LoRA

A test adapter for Qwen/Qwen3.8-27B trained to wrap answers to ordinary prompts in a fixed JSON envelope. The prompt does not need to request JSON. This is intended for testing obvious adapter behavior rather than response quality or strict schema guarantees.

Example

Prompt: Explain why the sky appears blue in two short sentences.

Output:

{"adapter":"json","answer":"The sky appears blue because the molecules in the atmosphere scatter blue light more than other colors. This scattering of blue light is known as Rayleigh scattering."}

Training

• Dataset: HuggingFaceH4/ultrachat_200k (train_sft split)
• Generic user prompts; assistant answers wrapped during preprocessing
• 200 optimizer steps; maximum sequence length 1,024
• LoRA rank 8, alpha 16, dropout 0.05
• Targets the Qwen projection layers and lm_head
• 60,391,424 trainable parameters

Loading

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "Qwen/Qwen3.8-27B"
tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(
    base_id, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(
    base, "William-Gao1/qwen3.8-27b-json-lora"
)