Downloads · 30 days
26
7% of all-time downloads
chanceQZhang/zhihu-tech-zhichang-qwen-7b
zhihu-tech-zhichang-qwen-7b is a text generation model from chanceQZhang. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as gpl-3.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
26
7% of all-time downloads
All-time downloads
352
Public
Repo size
36.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf20.7 GB · 57%
From the Hugging Face model README
axolotl version: 0.13.0.dev0
# config_sft_zhihu.yml
base_model: Orion-zhen/Meissa-Qwen2.5-7B-Instruct
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
# 使用您上传的数据集
datasets:
- path: chanceQZhang/zhihuhighvotes
type: chat_template # ChatML 格式使用 chat_template
split: train
# 提速核心
sample_packing: true
pad_to_sequence_len: true
# LoRA 配置
adapter: lora
lora_r: 8
lora_alpha: 32
lora_dropout: 0.1
lora_target_modules:
- q_proj
- v_proj
- k_proj
- o_proj
- gate_proj
- up_proj
- down_proj
# --- 核心优化:显存节省配置 ---
bf16: true # 30/40系列或A系列显卡必开,提升速度且省显存
fp16: false
gradient_checkpointing: true # 必开!用计算时间换空间,大幅降低显存占用
flash_attention: true # 必开!大幅降低长文本下的显存需求
# 训练配置
sequence_len: 2048
micro_batch_size: 6
gradient_accumulation_steps: 3
num_epochs: 2
learning_rate: 0.00005
# 减少中间开销
logging_steps: 10
eval_steps: 100
save_steps: 302
# 输出
output_dir: ./outputs/zhihu-tech-career-lora
</details><br>
This model is a fine-tuned version of Orion-zhen/Meissa-Qwen2.5-7B-Instruct on the chanceQZhang/zhihuhighvotes dataset.
More information needed
More information needed
More information needed
The following hyperparameters were used during training: