guancha-qwen

Qwen3-4B-Instruct-2507-Guancha-LoRA | Qwen3-4B 观察者网 LoRA 适配器

English | 中文

---

<a name="english"></a>

🇬🇧 English

Model Description

This is a LoRA adapter fine-tuned on the Guancha.cn (观察者网) news dataset, based on Qwen/Qwen3-4B-Instruct-2507. The model uses LoRA (Low-Rank Adaptation) technology to enable the base model to answer political and current affairs questions in the analytical style of Guancha.cn, while retaining the general capabilities of the original model.

Key Information

• Base Model: Qwen/Qwen3-4B-Instruct-2507
• Fine-tuning Method: LoRA (rank=16, alpha=32, dropout=0.05)
• Training Data: Guancha.cn news Q&A dataset (~1,232 samples)
• Training Framework: LLaMA-Factory
• Model Type: LoRA Adapter (requires base model)

Intended Uses

This model is suitable for:

• Political News Analysis: Analyzing and commenting on current political events
• International Relations Q&A: Answering questions about international relations
• Journalistic Style Generation: Generating responses with objective, professional, and globally-minded journalistic style

Limitations

• The model is trained on a specific news media style and may exhibit biases in certain domains
• Limited training data (~1,200 samples) may result in incomplete coverage of some topics
• Do not use model outputs as factual evidence; use them only as references

How to Use

⚠️ Important: This is a LoRA adapter, not a complete model. You need to load it with the base model.

Method 1: Using PEFT Library (Recommended)

from transformers import AutoTokenizer
from peft import AutoPeftModelForCausalLM
import torch

# Load the adapter (automatically merges with base model)
model = AutoPeftModelForCausalLM.from_pretrained(
    "galoyoung/guancha-qwen",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "galoyoung/guancha-qwen",
    trust_remote_code=True
)

# Use the model
messages = [{"role": "user", "content": "Why did Macron visit China in December?"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.8,
        do_sample=True
    )

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[1]:], 
    skip_special_tokens=True
)
print(response)

Method 2: Using LLaMA-Factory

llamafactory-cli chat \
    --model_name_or_path Qwen/Qwen3-4B-Instruct-2507 \
    --adapter_name_or_path galoyoung/guancha-qwen

❌ Incorrect Usage

# This won't work! This is an adapter, not a complete model
from transformers import pipeline
pipe = pipeline("text-generation", model="galoyoung/guancha-qwen")  # ❌ Will error

Training Details

Training Hyperparameters

• Learning Rate: 2e-05
• Train Batch Size: 1
• Gradient Accumulation Steps: 8
• Total Train Batch Size: 8
• Epochs: 5.0
• Optimizer: adamw_torch (betas=(0.9,0.999), epsilon=1e-08)
• LR Scheduler: cosine (warmup_ratio=0.03)

Framework Versions

• PEFT 0.17.1
• Transformers 4.57.3
• PyTorch 2.3.1+cu121
• Datasets 4.0.0
• Tokenizers 0.22.1

Training Data Source

The training data comes from Q&A pairs collected from the Guancha.cn Official Zhihu Account, crawled and organized using the open-source tool MediaCrawler. The dataset has been cleaned and formatted, converted to the standard ChatML format.

Citation

If you use this model in your research, please cite:

@misc{guancha-qwen-lora,
  title={Qwen3-4B-Instruct-2507-Guancha-LoRA},
  author={galoyoung},
  year={2025},
  howpublished={\url{https://huggingface.co/galoyoung/guancha-qwen}}
}

License

This model is released under the same license as the base model Qwen/Qwen3-4B-Instruct-2507.

Disclaimer

This model is for research and educational purposes only. The training data content belongs to the original author (Guancha.cn) and the platform (Zhihu). Please comply with relevant terms of service and copyright laws when using this model.

---

<a name="chinese"></a>

🇨🇳 中文

模型简介

这是一个基于 Qwen/Qwen3-4B-Instruct-2507 使用**观察者网（Guancha.cn）**新闻数据集进行微调的 LoRA 适配器模型。本模型采用 LoRA（Low-Rank Adaptation）技术，使基础模型能够以"观察者网"的新闻分析风格回答时政问题，同时保持原模型的通用能力。

关键信息

• 基础模型： Qwen/Qwen3-4B-Instruct-2507
• 微调方法： LoRA (rank=16, alpha=32, dropout=0.05)
• 训练数据： 观察者网新闻问答数据集（约 1,232 条样本）
• 训练框架： LLaMA-Factory
• 模型类型： LoRA 适配器（需要配合基础模型使用）

适用场景

本模型适用于：

• 时政新闻分析： 分析和评论当前政治事件
• 国际关系问答： 回答国际关系相关问题
• 新闻风格生成： 生成客观、专业、具有国际视野的新闻风格回答

局限性

• 本模型基于特定新闻媒体的风格训练，可能在某些领域存在偏见
• 训练数据规模较小（约 1,200 条），可能在某些话题上表现不够全面
• 请勿将模型输出作为事实依据，仅作为参考

使用方法

⚠️ 重要提示： 这是一个 LoRA 适配器，不是完整的模型。使用时需要配合基础模型加载。

方法一：使用 PEFT 库（推荐）

from transformers import AutoTokenizer
from peft import AutoPeftModelForCausalLM
import torch

# 加载适配器（会自动合并基础模型）
model = AutoPeftModelForCausalLM.from_pretrained(
    "galoyoung/guancha-qwen",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained(
    "galoyoung/guancha-qwen",
    trust_remote_code=True
)

# 使用模型
messages = [{"role": "user", "content": "法国总统马克龙将于 12 月初访华，为何会在此时访华？"}]
prompt = tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.7,
        top_p=0.8,
        do_sample=True
    )

response = tokenizer.decode(
    outputs[0][inputs.input_ids.shape[1]:], 
    skip_special_tokens=True
)
print(response)

方法二：使用 LLaMA-Factory

llamafactory-cli chat \
    --model_name_or_path Qwen/Qwen3-4B-Instruct-2507 \
    --adapter_name_or_path galoyoung/guancha-qwen

❌ 错误用法

# 这样不行！因为这是一个适配器，不是完整模型
from transformers import pipeline
pipe = pipeline("text-generation", model="galoyoung/guancha-qwen")  # ❌ 会报错

训练详情

训练超参数

• 学习率： 2e-05
• 训练批次大小： 1
• 梯度累积步数： 8
• 总训练批次大小： 8
• 训练轮数： 5.0
• 优化器： adamw_torch (betas=(0.9,0.999), epsilon=1e-08)
• 学习率调度器： cosine (warmup_ratio=0.03)

框架版本

• PEFT 0.17.1
• Transformers 4.57.3
• PyTorch 2.3.1+cu121
• Datasets 4.0.0
• Tokenizers 0.22.1

训练数据来源

训练数据来源于从观察者网知乎官方账号收集的问答对，通过开源工具 MediaCrawler 爬取并整理获得。数据集经过清洗和格式化处理，转换为标准的 ChatML 格式。

引用

如果您在研究中使用此模型，请引用：

@misc{guancha-qwen-lora,
  title={Qwen3-4B-Instruct-2507-Guancha-LoRA},
  author={galoyoung},
  year={2025},
  howpublished={\url{https://huggingface.co/galoyoung/guancha-qwen}}
}

许可证

本模型遵循与基础模型 Qwen/Qwen3-4B-Instruct-2507 相同的许可证。

免责声明

本模型仅供学术研究和教育用途。训练数据内容的版权归原作者（观察者网）及平台（知乎）所有。在使用本模型时，请遵守相关的服务条款及版权法律法规。

---

license: other base_model: Qwen/Qwen3-4B-Instruct-2507 library_name: peft tags:

• base_model:adapter:Qwen/Qwen3-4B-Instruct-2507
• llama-factory
• lora
• transformers
• guancha
• news
• chinese pipeline_tag: text-generation language:
• zh
• en task_categories:
• text-generation size_categories:
• 1B<n<10B

---