Downloads · 30 days
3
25% of all-time downloads
emmixam/mistral-7b-v0-3-finetune-monbapteme
mistral-7b-v0-3-finetune-monbapteme is a machine learning model from emmixam. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for peft. The card lists the license as apache-2.0.
should probably proofread and complete it, then remove this comment. --
Downloads · 30 days
3
25% of all-time downloads
All-time downloads
12
Public
Repo size
117 MB
Likes
0
Public
Click a slice to open those files.
.safetensors101 MB · 64%
From the Hugging Face model README
axolotl version: 0.8.0.dev0
base_model: TinyLlama/TinyLlama_v1.1
datasets:
- path: mhenrichsen/alpaca_2k_test
type: alpaca
output_dir: ./outputs/mymodel
sequence_len: 8192
adapter: lora
lora_r: 8
lora_alpha: 16
lora_dropout: 0.05
lora_target_modules:
- q_proj
- v_proj
- k_proj
- o_proj
- gate_proj
- down_proj
- up_proj
gradient_accumulation_steps: 1
micro_batch_size: 4
num_epochs: 3
optimizer: adamw_bnb_8bit
learning_rate: 2e-5
load_in_8bit: true
train_on_inputs: false
bf16: auto
</details><br>
This model is a fine-tuned version of TinyLlama/TinyLlama_v1.1 on the mhenrichsen/alpaca_2k_test dataset.
More information needed
More information needed
More information needed
The following hyperparameters were used during training: