Downloads · 30 days
2
9% of all-time downloads
philipperen55/Qwen2.5-14B-v3-3690-dataset52SFT
Qwen2.5-14B-v3-3690-dataset52SFT is a machine learning model from philipperen55. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
2
9% of all-time downloads
All-time downloads
23
Public
Repo size
2.8 GB
Likes
0
Public
Click a slice to open those files.
.bin551 MB · 49%
From the Hugging Face model README
axolotl version: 0.5.0
base_model: philipperen55/Qwen2.5-14B-style-MERGED-BF16-v3-3690
model_type: AutoModelForCausalLM
tokenizer_type: AutoTokenizer
trust_remote_code: true
load_in_8bit: false
load_in_4bit: false
datasets:
- path: philipperen55/dataset52SFT
ds_type: json
data_files: dataset52SFT.jsonl
type: input_output
train_on_inputs: false
add_eos_token: true
dataset_prepared_path: /workspace/prepared_data
val_set_size: 0.01
output_dir: /workspace/output
sequence_len: 2048
sample_packing: false
pad_to_sequence_len: false
group_by_length: true
special_tokens:
eos_token: "<|endoftext|>"
pad_token: "<|endoftext|>"
adapter: lora
lora_r: 32
lora_alpha: 64
lora_dropout: 0.05
lora_target_modules:
- q_proj
- k_proj
- v_proj
- o_proj
- gate_proj
- up_proj
- down_proj
gradient_accumulation_steps: 32
micro_batch_size: 2
num_epochs: 1
learning_rate: 1e-5
lr_scheduler: constant_with_warmup
warmup_ratio: 0.10
optimizer: adamw_torch_fused
weight_decay: 0.0
max_grad_norm: 1.0
bf16: true
fp16: false
tf32: true
flash_attention: true
sdp_attention: false
overrides_of_model_config:
attn_implementation: flash_attention_2 # forcer FA2 jusqu’au modèle
use_cache: false
gradient_checkpointing: false
# --- Reproductibilité ---
seed: 42
logging_steps: 5
save_steps: 20
eval_strategy: steps
eval_steps: 20
save_total_limit: 1
wandb_project: Qwen2.5-14B-v3-3690-dataset52SFT_00
hub_model_id: philipperen55/Qwen2.5-14B-v3-3690-dataset52SFT
push_to_hub: true
hub_strategy: every_save
</details><br>
This model is a fine-tuned version of philipperen55/Qwen2.5-14B-style-MERGED-BF16-v3-3690 on the None dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| No log | 0.0129 | 1 | 1.1625 |
| 1.2063 | 0.2586 | 20 | 1.1249 |
| 1.1581 | 0.5172 | 40 | 1.0926 |
| 1.146 | 0.7758 | 60 | 1.0773 |