Downloads · 30 days
10
4% of all-time downloads
DAMI-base-merge-0408
Onuii/DAMI-base-merge-0408
DAMI-base-merge-0408 is a text generation model from Onuii. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: Onuii - License: apache-2.0 - Finetuned from model : kakaocorp/kanana-nano-2.1b-base
All-time downloads
270
Public
Parameters
2.3B
4.7 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Apr 8, 2025
- Updated
- Apr 16, 2025
- SHA
- ab74a092
Try a prompt
Base models
Model card
From the Hugging Face model README
Uploaded model
-
Developed by: Onuii
-
License: apache-2.0
-
Finetuned from model : kakaocorp/kanana-nano-2.1b-base
--run_name "DAMI-base-merge-0408"
--model_name "Onuii/DAMI-base-merge-0408"
--dataset_name "Onuii/DAMI-Pretrain-Dataset-Tokenize8192"
--max_seq_length 8192
--dtype "bfloat16"
--load_in_4bit True
--r 128
--lora_alpha 32
--lora_dropout 0
--use_rslora True
--batch_size 32
--gradient_accumulation_steps 4
--learning_rate 1e-4
--embedding_learning_rate 1e-5
--warmup_ratio 0.1
--num_train_epochs 3
--optimizer "adamw_8bit"
--weight_decay 0.01
--lr_scheduler_type "linear"
--output_dir "../outputs"
--logging_dir "../logs"
--logging_steps 1
--save_steps 200
--save_total_limit 2
--report_to "wandb"
--use_gradient_checkpointing False
--save_strategy "steps"
--random_seed 562
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Apr 8, 2025
- Updated
- Apr 16, 2025