qwen2.5-7b-lora-4-bit

qwen2.5-7b-lora-4-bit

This is a LoRA fine-tuned model based on Qwen2.5-7B-Instruct, trained on WeChat chat records to learn a specific user's speaking style.

Model Information

• Base Model: Qwen/Qwen2.5-7B-Instruct
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Training Framework: Unsloth
• LoRA Rank: 8
• Training Data: WeChat chat records

Usage

Load Model

from unsloth import FastLanguageModel

# Load base model
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="Qwen/Qwen2.5-7B-Instruct",
    max_seq_length=1024,
    dtype=None,
    load_in_4bit=True,
)

# Load LoRA weights
from peft import PeftModel
model = PeftModel.from_pretrained(model, "chenyijiu/qwen2.5-7b-lora-4-bit")

# Enable inference mode
FastLanguageModel.for_inference(model)

# Generate response
inputs = tokenizer(
    ["<|im_start|>system\n你是一个抽象的助手。<|im_end|>\n<|im_start|>user\n你好<|im_end|>\n<|im_start|>assistant\n"],
    return_tensors="pt"
).to("cuda")

outputs = model.generate(**inputs, max_new_tokens=128, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Merge LoRA Weights (Optional)

# Merge LoRA weights into base model
model = model.merge_and_unload()

# Save full model
model.save_pretrained("merged_model")
tokenizer.save_pretrained("merged_model")

Training Parameters

• Max Sequence Length: 1024
• Batch Size: 1
• Gradient Accumulation Steps: 8
• Learning Rate: 2e-4
• Epochs: 3
• Optimizer: AdamW 8-bit

License

Apache 2.0

Citation

If you use this model, please cite:

@misc{qwen2.5_7b_lora_4_bit,
  author = {chenyijiu},
  title = {qwen2.5-7b-lora-4-bit},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/chenyijiu/qwen2.5-7b-lora-4-bit}}
}