Downloads · 30 days
11
18% of all-time downloads
theblackcat102/Qwen3-1.7B-nq-baseline-finetune-5epochs
Qwen3-1.7B-nq-baseline-finetune-5epochs is a text generation model from theblackcat102. Use it when you need the model to write or continue text. It is set up for transformers.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
11
18% of all-time downloads
All-time downloads
60
Public
Parameters
1.7B
4.1 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors4.1 GB · 100%
How the weights are stored.
BF161.4B · 82%
From the Hugging Face model README
axolotl version: 0.10.0
base_model: Abner0803/Qwen3-1.7B-nq-text-100k-with_pseudo_queries
datasets:
- path: Abner0803/nq_text-with_pseudo_query-100k-gr
data_files: data/icl_test.jsonl
type: chat_template
chat_template: tokenizer_default_fallback_chatml
field_messages: conversations
message_property_mappings:
role: role
content: content
roles:
assistant:
- assistant
- gpt
- model
user:
- user
- human
system:
- system
roles_to_train: ["assistant"]
train_on_eos: "turn"
dataset_processes: 6
streaming: false
shuffle_merged_datasets: true
output_dir: /home/theblackcat/ICLGR/checkpoints/Qwen3-1.7B-nq-baseline-finetune-5epochs
sequence_len: 2048
sample_packing: false
flash_attention: false
xformers_attention: false
flex_attention: false
sdp_attention: true
overrides_of_model_config:
_attn_implementation: "sdpa"
gradient_accumulation_steps: 128
micro_batch_size: 1
num_epochs: 5
optimizer: adamw_torch
lr_scheduler: cosine
learning_rate: 0.0001
warmup_ratio: 0.1
weight_decay: 0.0
bf16: true
tf32: false
gradient_checkpointing: true
logging_steps: 50
save_strategy: steps
save_steps: 100
save_total_limit: 3
special_tokens:
eos_token: "<|im_end|>"
val_set_size: 0.0
wandb_project: ICLGR-NQ
wandb_entity: abnerden0803-national-taiwan-university
wandb_watch:
wandb_name: qwen3-1.7b-nq-baseline-direct-finetune-5epochs
wandb_log_model:
</details><br>
This model is a fine-tuned version of Abner0803/Qwen3-1.7B-nq-text-100k-with_pseudo_queries on the Abner0803/nq_text-with_pseudo_query-100k-gr dataset.
More information needed
More information needed
More information needed
The following hyperparameters were used during training: