Downloads · 30 days
19
3% of all-time downloads
llama-3-8b-samantha
ruslandev/llama-3-8b-samantha
llama-3-8b-samantha is a text generation model from ruslandev. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
- Developed by: ruslandev - License: apache-2.0 - Finetuned from model : unsloth/llama-3-8b-bnb-4bit
All-time downloads
546
Public
Parameters
8B
16.6 GB on disk
Likes
3
Public
Repo makeup
Click a slice to open those files.
.safetensors16.2 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Apr 26, 2024
- Updated
- Apr 29, 2024
- SHA
- a803efac
Try a prompt
Base models
Model card
From the Hugging Face model README
Uploaded model
- Developed by: ruslandev
- License: apache-2.0
- Finetuned from model : unsloth/llama-3-8b-bnb-4bit
This model is finetuned on the data of Samantha.
Prompt format is Alpaca. I used the same system prompt as the original Samantha.
"""Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.
### Instruction:
{SYSTEM_PROMPT}
### Input:
{QUESTION}
### Response:
"""
Training
gptchain framework has been used for training.
Training hyperparameters
- learning_rate: 2e-4
- seed: 3407
- gradient_accumulation_steps: 4
- per_device_train_batch_size: 2
- optimizer: adamw_8bit
- lr_scheduler_type: linear
- warmup_steps: 5
- num_epochs: 2
- weight_decay: 0.01
Training results
| Training Loss | Epoch | Step |
|---|---|---|
| 2.0778 | 0.0 | 1 |
| 0.6255 | 0.18 | 120 |
| 0.6208 | 0.94 | 620 |
| 0.6244 | 2.0 | 1306 |
2 epoch finetuning from llama-3-8b took 1 hour on a single A100 with Unsloth and Huggingface's TRL library.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en
- Created
- Apr 26, 2024
- Updated
- Apr 29, 2024