Downloads · 30 days
37
41% of all-time downloads
AlekseyKorshuk/ai-detection-gutenberg-human-formatted-ai-v1-sft-qwen-7b
ai-detection-gutenberg-human-formatted-ai-v1-sft-qwen-7b is a text generation model from AlekseyKorshuk. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
37
41% of all-time downloads
All-time downloads
90
Public
Parameters
7.6B
40.3 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors36.6 GB · 100%
From the Hugging Face model README
axolotl version: 0.4.1
base_model: Qwen/Qwen2.5-7B-Instruct
strict: false
plugins:
- axolotl.integrations.liger.LigerPlugin
liger_rope: true
liger_rms_norm: true
liger_swiglu: true
liger_fused_linear_cross_entropy: true
chat_template: chatml
datasets:
- path: AlekseyKorshuk/ai-detection-gutenberg-human-formatted-ai-v1-sft
type: chat_template
field_messages: messages
message_field_role: role
message_field_content: content
roles:
user:
- user
assistant:
- assistant
val_set_size: 0.05
output_dir: ./outputs/out
eval_table_size: 0
eval_max_new_tokens: 1
sequence_len: 16384
sample_packing: false
eval_sample_packing: false
pad_to_sequence_len: false
wandb_project: ai-seo-rewriter
wandb_entity:
wandb_watch:
wandb_name: ai-detection-gutenberg-human-formatted-ai-v1-sft-qwen-7b
wandb_log_model:
gradient_accumulation_steps: 2
micro_batch_size: 16
eval_batch_size: 16
num_epochs: 1
optimizer: adamw_torch
# adam_beta1: 0.9
# adam_beta2: 0.95
max_grad_norm: 1.0
# adam_epsilon: 0.00001
lr_scheduler: cosine
learning_rate: 1e-5
train_on_inputs: false
group_by_length: false
bf16: true
fp16: false
tf32: true
gradient_checkpointing: true
gradient_checkpointing_kwargs:
use_reentrant: false
early_stopping_patience:
resume_from_checkpoint:
logging_steps: 1
xformers_attention:
flash_attention: true
warmup_ratio: 0.1
evals_per_epoch: 10
saves_per_epoch: 1
debug:
deepspeed:
weight_decay: 0.0
fsdp:
- full_shard
- auto_wrap
fsdp_config:
fsdp_limit_all_gathers: true
fsdp_sync_module_states: true
fsdp_offload_params: true
fsdp_use_orig_params: false
fsdp_cpu_ram_efficient_loading: true
fsdp_auto_wrap_policy: TRANSFORMER_BASED_WRAP
fsdp_transformer_layer_cls_to_wrap: Qwen2DecoderLayer
fsdp_state_dict_type: FULL_STATE_DICT
hub_model_id: AlekseyKorshuk/ai-detection-gutenberg-human-formatted-ai-v1-sft-qwen-7b
</details><br>
This model is a fine-tuned version of Qwen/Qwen2.5-7B-Instruct on the None dataset. It achieves the following results on the evaluation set:
More information needed
More information needed
More information needed
The following hyperparameters were used during training:
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 4.3587 | 0.0010 | 1 | 2.1768 |
| 1.9887 | 0.1005 | 96 | 0.9925 |
| 1.8923 | 0.2010 | 192 | 0.9727 |
| 1.7986 | 0.3016 | 288 | 0.9596 |
| 1.9068 | 0.4021 | 384 | 0.9490 |
| 1.8078 | 0.5026 | 480 | 0.9395 |
| 1.7818 | 0.6031 | 576 | 0.9326 |
| 1.8066 | 0.7037 | 672 | 0.9264 |
| 1.7729 | 0.8042 | 768 | 0.9225 |
| 1.8047 | 0.9047 | 864 | 0.9207 |