Qwen3.8-27B-DeepResearch

Qwen3.8-27B-DeepResearch

Qwen3.8-27B-DeepResearch is a specialized foundation model post-trained for deep scientific reasoning, autonomous literature search, evidence synthesis, and citation-grounded question answering.

Post-trained using a Hybrid Supervised Fine-Tuning (SFT) + Direct Preference Optimization (DPO) pipeline built on top of Qwen/Qwen3.8-27B.

Key Capabilities

1. Structured Multi-Step Reasoning (<thought>): Employs deep chain-of-thought hypothesis testing, error correction, and mathematical derivation before formulating responses.
2. Strict Citation Grounding ([1], [2]): Trained on multi-document retrieval benchmarks (AVeriTeC, ALCE, SciFact) to eliminate citation hallucination and tie claims directly to verified text spans.
3. Academic & Scientific Depth: Excels in ingesting and synthesizing literature across physics, biomedicine, chemistry, and computer science.
4. General Helpfulness & Alignment: Tuned with HelpSteer2 multi-attribute preferences to ensure high clarity, minimal verbosity, and zero sycophancy.

---

Prompt Format

Qwen3.8-27B-DeepResearch follows the standard ChatML structure with an internal <thought> reasoning trace:

<|im_start|>system
You are a deeply researchful, rigorous, and helpful AI assistant. Always reason thoroughly inside <thought>...</thought> before delivering factual, structured responses with precise citations [1], [2].
<|im_end|>
<|im_start|>user
Compare the mechanisms of mRNA-1273 and BNT162b2 lipid nanoparticles in cellular uptake, citing relevant literature.
<|im_end|>
<|im_start|>assistant
<thought>
Deconstruct lipid nanoparticle formulations:
- mRNA-1273 (Moderna): SM-102, DSPC, cholesterol, PEG2000-DMG (50:10:38.5:1.5 molar ratio).
- BNT162b2 (Pfizer/BioNTech): ALC-0315, DSPC, cholesterol, ALC-0159 (46.3:9.4:42.7:1.6 molar ratio).
Examine endocytosis pathway: macropinocytosis vs clathrin-mediated uptake, endosomal escape kinetics.
Formulate grounded response with clear citations.
</thought>
Both mRNA-1273 and BNT162b2 deliver nucleoside-modified mRNA encapsulated within ionizable lipid nanoparticles (LNPs) [1]...

---

Inference with Transformers & vLLM

Using Hugging Face Transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen3.8-27B"
adapter_id = "honvert/Qwen3.8-27B-DeepResearch"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)

messages = [
    {"role": "system", "content": "You are a deeply researchful and helpful AI assistant."},
    {"role": "user", "content": "Synthesize recent findings on superconducting hydride materials at megabar pressures."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

Training Details

• Base Model: Qwen/Qwen3.8-27B (Apache 2.0)
• Methodology: LoRA (Rank 64, Alpha 128, all linear layers)
• Dataset Recipe:
  • 35% Deep Reasoning & Chain-of-Thought (OpenThoughts3, OpenR1-Math)
  • 30% Web Search, Tool Execution & Citations (AVeriTeC, ALCE, FactCheck)
  • 20% Scientific Paper Synthesis (SciFact, PubMedQA, S2ORC)
  • 15% Multi-Attribute Helpfulness (HelpSteer2)
• License: Apache 2.0