Downloads · 30 days
13
23% of all-time downloads
SuperPuperD/yurteg-qwen2.5-1.5b-sft-v2
yurteg-qwen2.5-1.5b-sft-v2 is a text generation model from SuperPuperD. Use it when you need the model to write or continue text. It is set up for peft. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
13
23% of all-time downloads
All-time downloads
56
Public
Repo size
1.4 GB
Likes
0
Public
Click a slice to open those files.
.safetensors1.2 GB · 70%
From the Hugging Face model README
axolotl version: 0.15.0
# ============================================================
# ЮрТэг — SFT v2 Fine-Tuning Config для RunPod/Axolotl
# Модель: Qwen 2.5 1.5B Instruct
# Датасет: SuperPuperD/yurteg-legal-sft (660 train + 95 val)
# Формат: Qwen-native tool call (<tool_call>...</tool_call>)
# Схема: 13 полей (без confidence, + payment_*)
# ============================================================
# --- Модель ---
base_model: Qwen/Qwen2.5-1.5B-Instruct
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
# --- Квантизация (QLoRA 4-bit) ---
load_in_8bit: false
load_in_4bit: true
# --- Датасет ---
datasets:
- path: SuperPuperD/yurteg-legal-sft
type: chat_template
chat_template: tokenizer_default
field_messages: messages
roles_to_train:
- assistant
train_on_eos: last
data_files:
- train.jsonl
# --- Валидация (отдельный файл) ---
val_set_size: 0
val_datasets:
- path: SuperPuperD/yurteg-legal-sft
type: chat_template
chat_template: tokenizer_default
field_messages: messages
data_files:
- val.jsonl
# --- Результат ---
output_dir: ./outputs/yurteg-qwen2.5-1.5b-sft-v2
# --- LoRA (максимальное качество) ---
adapter: qlora
lora_r: 64
lora_alpha: 128
lora_dropout: 0.05
lora_target_linear: true
# --- Длина и упаковка ---
sequence_len: 4096
sample_packing: true
pad_to_sequence_len: true
train_on_inputs: false
# --- Обучение (агрессивные настройки) ---
micro_batch_size: 2
gradient_accumulation_steps: 8
num_epochs: 8
optimizer: adamw_bnb_8bit
lr_scheduler: cosine
learning_rate: 0.0001
warmup_ratio: 0.1
weight_decay: 0.01
# --- Точность ---
bf16: auto
tf32: true
# --- Оптимизация памяти ---
gradient_checkpointing: true
flash_attention: true
# --- Логирование и сохранение ---
logging_steps: 5
evals_per_epoch: 4
saves_per_epoch: 1
save_total_limit: 3
# --- Воспроизводимость ---
seed: 42
# --- Специальные токены (Qwen) ---
special_tokens:
pad_token: "<|endoftext|>"
</details><br>
This model is a fine-tuned version of Qwen/Qwen2.5-1.5B-Instruct on the SuperPuperD/yurteg-legal-sft dataset.
More information needed
More information needed
More information needed
The following hyperparameters were used during training: