deepseek_wh2

DeepSeek-R1 Chinese Multiple Choice Reasoning Model

<!-- Provide a quick summary of what the model is/does. -->

This model is a LoRA fine-tuned version of DeepSeek-R1-Distill-Qwen-7B specifically trained for Chinese multiple choice question answering with reasoning capabilities.

Model Details

Model Description

<!-- Provide a longer summary of what this model is. -->

This model has been fine-tuned using LoRA (Low-Rank Adaptation) on Chinese multiple choice questions with detailed reasoning explanations. The model is designed to understand Chinese questions, analyze multiple options (A, B, C, D), and provide reasoned answers with explanations.

• Developed by: jasmine313112031 (YANG YING HSUAN)
• Model type: Causal Language Model with LoRA adapter
• Language(s) (NLP): Chinese
• License: Apache 2.0
• Finetuned from model: unsloth/DeepSeek-R1-Distill-Qwen-7B

Model Sources

• Repository: GitHub Repository
• Training Logs: W&B Dashboard

Uses

Direct Use

This model is designed for Chinese multiple choice question answering tasks. It can:

• Answer multiple choice questions in Chinese
• Provide reasoning explanations for answers
• Handle questions across various domains

Downstream Use

The model can be integrated into:

• Educational assessment systems
• Chinese language learning platforms
• Automated grading systems
• Reasoning evaluation benchmarks

Out-of-Scope Use

This model should not be used for:

• Tasks requiring factual accuracy beyond training data cutoff
• High-stakes decision making without human oversight
• Generating harmful or biased content

Bias, Risks, and Limitations

Known Issues:

• Strong bias towards option A (97.6% of predictions in test set)
• May reflect training data distribution imbalance
• Limited to multiple choice format with 4 options

Recommendations

Users should be aware of the model's bias towards option A and consider:

• Randomizing option order during inference
• Using ensemble methods to reduce bias
• Validating outputs against ground truth when available

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained("unsloth/DeepSeek-R1-Distill-Qwen-7B")
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/DeepSeek-R1-Distill-Qwen-7B",
    torch_dtype=torch.float16,
    device_map="auto"
)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "jasmine313112031/deepseek_wh2")

# Example usage
prompt = """問題：六四事件發生在哪一年？
選項：
A. 1989年
B. 1990年
C. 1991年
D. 1992年

請從上面的選項中選出最正確的答案。
你的答案必須使用「正確答案：X」的格式，X是A、B、C或D中的一個。

思考過程："""

inputs = tokenizer(prompt, return_tensors="pt")
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=100, do_sample=False)
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)