Downloads · 30 days
0
tatasauce4life/qwen3-4b-en2zh-orpo_beta0.2
qwen3-4b-en2zh-orpo_beta0.2 is a translation model from tatasauce4life. Use it when you need text moved from one language to another. The card lists the license as apache-2.0.
- 模型類型: LoRA Adapter (SFT + ORPO 累積更新) - 基礎模型: unsloth/Qwen3-4B-Base-unsloth-bnb-4bit - SFT 模型來源: tatasauce4life/qwen3-4b-en2zh-lora - 訓練方法: ORPO (Odds Ratio Preference Optimization) - 任務: 英文到中文小說翻譯
Downloads · 30 days
0
Access
Public
Updated Sep 19, 2026
Repo size
276 MB
Likes
0
Public
Click a slice to open those files.
.safetensors264 MB · 96%
From the Hugging Face model README
unsloth/Qwen3-4B-Base-unsloth-bnb-4bittatasauce4life/qwen3-4b-en2zh-lora{
"prompt": "英文原文",
"chosen": "人工譯文(高品質)",
"rejected": "機器翻譯(低品質)"
}
### English
{english_text}
### Chinese
{chinese_translation}
| 參數 | 值 |
|---|---|
| ORPO beta | 0.1 |
| Learning Rate | 0.0001 |
| Epochs | 5 |
| Batch Size | 1 |
| Gradient Accumulation | 16 |
| Max Sequence Length | 1024 |
| LoRA r | 16 |
| LoRA alpha | 32 |
| Epoch | Training Loss | Validation Loss | Runtime | Samples Per Second | Steps Per Second | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | Nll Loss | Log Odds Ratio | Log Odds Chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1.844135 | 1.854190 | 87.344000 | 4.167000 | 4.167000 | -0.131862 | -0.143242 | 0.450549 | 0.011379 | -1.432417 | -1.318622 | 7.127260 | 7.001682 | 1.778190 | -0.759993 | 0.166458 |
| 2 | 1.552538 | 1.815805 | 87.031400 | 4.182000 | 4.182000 | -0.129877 | -0.175067 | 0.620879 | 0.045190 | -1.750668 | -1.298767 | 5.576954 | 5.433165 | 1.753491 | -0.623141 | 0.613916 |
| 3 | 1.331949 | 1.891158 | 87.251700 | 4.172000 | 4.172000 | -0.141172 | -0.201667 | 0.642857 | 0.060495 | -2.016670 | -1.411715 | 4.119596 | 3.943425 | 1.829881 | -0.612768 | 0.802193 |
| 4 | 1.110539 | 2.014756 | 86.803400 | 4.193000 | 4.193000 | -0.155560 | -0.229186 | 0.670330 | 0.073625 | -2.291857 | -1.555604 | 3.257812 | 3.058837 | 1.955649 | -0.591075 | 0.954718 |
| 5 | 1.028039 | 2.102785 | 86.433800 | 4.211000 | 4.211000 | -0.164843 | -0.241751 | 0.673077 | 0.076909 | -2.417511 | -1.648425 | 2.883676 | 2.680442 | 2.043374 | -0.594112 | 0.977506 |
此模型是 LoRA adapter,包含 SFT + ORPO 的累積更新。
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="tatasauce4life/qwen3-4b-en2zh-orpo_beta0.2",
max_seq_length=1024,
load_in_4bit=True,
)
FastLanguageModel.for_inference(model)
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model = AutoPeftModelForCausalLM.from_pretrained(
"tatasauce4life/qwen3-4b-en2zh-orpo_beta0.2",
load_in_4bit=True,
)
tokenizer = AutoTokenizer.from_pretrained("tatasauce4life/qwen3-4b-en2zh-orpo_beta0.2")
注意: 此 adapter 的 base model 為
unsloth/Qwen3-4B-Base-unsloth-bnb-4bit。 adapter 包含 SFT 階段與 ORPO 階段的累積權重更新。
prompt = "### English\n{your_english_text}\n### Chinese\n"
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
result = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(result)