Downloads · 30 days
0
tatasauce4life/qwen3-4b-en2zh-orpo_beta0.4
qwen3-4b-en2zh-orpo_beta0.4 is a translation model from tatasauce4life. Use it when you need text moved from one language to another. The card lists the license as apache-2.0.
- 模型類型: LoRA Adapter (SFT + ORPO 累積更新) - 基礎模型: unsloth/Qwen3-4B-Base-unsloth-bnb-4bit - SFT 模型來源: tatasauce4life/qwen3-4b-en2zh-lora - 訓練方法: ORPO (Odds Ratio Preference Optimization) - 任務: 英文到中文小說翻譯
Downloads · 30 days
0
Access
Public
Updated Sep 21, 2026
Repo size
276 MB
Likes
0
Public
Click a slice to open those files.
.safetensors264 MB · 96%
From the Hugging Face model README
unsloth/Qwen3-4B-Base-unsloth-bnb-4bittatasauce4life/qwen3-4b-en2zh-lora{
"prompt": "英文原文",
"chosen": "人工譯文(高品質)",
"rejected": "機器翻譯(低品質)"
}
### English
{english_text}
### Chinese
{chinese_translation}
| 參數 | 值 |
|---|---|
| ORPO beta | 0.08 |
| Learning Rate | 5e-05 |
| Epochs | 4 |
| Batch Size | 1 |
| Gradient Accumulation | 16 |
| Max Sequence Length | 1024 |
| LoRA r | 16 |
| LoRA alpha | 32 |
| Epoch | Training Loss | Validation Loss | Runtime | Samples Per Second | Steps Per Second | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | Nll Loss | Log Odds Ratio | Log Odds Chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 1.858179 | 1.864919 | 93.603900 | 3.889000 | 3.889000 | -0.106697 | -0.099357 | 0.354396 | -0.007340 | -1.241965 | -1.333712 | 7.605855 | 7.485754 | 1.795982 | -0.861723 | -0.100940 |
| 2 | 1.666296 | 1.817527 | 92.809200 | 3.922000 | 3.922000 | -0.103970 | -0.114902 | 0.489011 | 0.010932 | -1.436270 | -1.299626 | 6.519071 | 6.387166 | 1.757984 | -0.744286 | 0.218549 |
| 3 | 1.562287 | 1.833609 | 94.846200 | 3.838000 | 3.838000 | -0.107248 | -0.130436 | 0.543956 | 0.023189 | -1.630455 | -1.340596 | 5.412767 | 5.248291 | 1.778783 | -0.685330 | 0.427241 |
| 4 | 1.482364 | 1.859507 | 93.639700 | 3.887000 | 3.887000 | -0.110568 | -0.136780 | 0.552198 | 0.026211 | -1.709749 | -1.382105 | 4.884839 | 4.700651 | 1.805095 | -0.680145 | 0.471212 |
此模型是 LoRA adapter,包含 SFT + ORPO 的累積更新。
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name="tatasauce4life/qwen3-4b-en2zh-orpo_beta0.4",
max_seq_length=1024,
load_in_4bit=True,
)
FastLanguageModel.for_inference(model)
from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer
model = AutoPeftModelForCausalLM.from_pretrained(
"tatasauce4life/qwen3-4b-en2zh-orpo_beta0.4",
load_in_4bit=True,
)
tokenizer = AutoTokenizer.from_pretrained("tatasauce4life/qwen3-4b-en2zh-orpo_beta0.4")
注意: 此 adapter 的 base model 為
unsloth/Qwen3-4B-Base-unsloth-bnb-4bit。 adapter 包含 SFT 階段與 ORPO 階段的累積權重更新。
prompt = "### English\n{your_english_text}\n### Chinese\n"
inputs = tokenizer(prompt, return_tensors="pt", add_special_tokens=False).to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, do_sample=False)
result = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(result)