Downloads · 30 days
10
2% of all-time downloads
laion/Sera-4.6-Lite-T2-v4-316-axolotl__Qwen3-8B-v3
Sera-4.6-Lite-T2-v4-316-axolotl__Qwen3-8B-v3 is a text generation model from laion. Use it when you need the model to write or continue text. It is set up for transformers.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
10
2% of all-time downloads
All-time downloads
510
Public
Parameters
8.2B
16.4 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors16.4 GB · 100%
From the Hugging Face model README
axolotl version: 0.16.0.dev0
# Sera v4 retrain v3 — size=316, num_epochs doubled 3->6
# v2 (num_epochs=3, tokenizer_default) recovered structure but model still
# emits malformed JSON ({"name": "view"}}} — 3 closing braces) and collapses
# str_replace_editor -> inner arguments.command='view'. Training data confirmed
# clean (5 rows / 140 tool calls / 0 malformed / tool names str_replace_editor/
# bash/submit). Hypothesis: ~120 grad updates at size=316 is too few to latch
# onto nested JSON structure + tool-name dictionary; double passes to 6 epochs.
base_model: Qwen/Qwen3-8B
deepspeed: /e/scratch/jureap59/feuer1/code/axolotl/deepspeed_configs/zero3_bf16.json
load_in_8bit: false
load_in_4bit: false
chat_template: tokenizer_default
# datasets: (stripped — axolotl embedded invalid local path)
_datasets_:
- laion/Sera-4.6-Lite-T2-v4-316
type: chat_template
field_messages: messages
ds_type: json
message_field_training: train
dataset_prepared_path: /e/data1/datasets/playground/ot-baf/axolotl_dataset_cache/sera-v4-316-v3
output_dir: /e/data1/datasets/playground/ot-baf/checkpoints/sera-v4-316-axolotl__Qwen3-8B-v3
sequence_len: 32768
wandb_project:
wandb_entity:
wandb_watch:
wandb_name: sera-v4-316-axolotl__Qwen3-8B-v3
wandb_log_model:
gradient_accumulation_steps: 8
micro_batch_size: 1
num_epochs: 6
optimizer: adamw_torch
lr_scheduler: cosine
learning_rate: 1e-5
adam_beta1: 0.9
adam_beta2: 0.95
bf16: auto
tf32: false
gradient_checkpointing: true
activation_offloading: true
resume_from_checkpoint:
logging_steps: 1
flash_attention: true
loss_watchdog_threshold: 5.0
loss_watchdog_patience: 3
warmup_ratio: 0.1875
evals_per_epoch: 0
save_strategy: epoch
weight_decay: 0.01
max_grad_norm: 1.0
special_tokens:
</details><br>
More information needed
More information needed
More information needed
The following hyperparameters were used during training: