Downloads · 30 days
18
5% of all-time downloads
Viking-SlimSonnet-v0.2-7B
mpasila/Viking-SlimSonnet-v0.2-7B
Viking-SlimSonnet-v0.2-7B is a text generation model from mpasila. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is a test model because the previous attempt failed. So it turns out I also trained this incorrectly due to the dataset using wrongly formatted ShareGPT so after I trained it again "correctly" for another full ep…
All-time downloads
332
Public
Parameters
7.6B
15.1 GB on disk
Likes
2
Public
Repo makeup
Click a slice to open those files.
.safetensors15.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Aug 25, 2024
- Updated
- Aug 28, 2024
- SHA
- 8624ed09
Try a prompt
Base models
Model card
From the Hugging Face model README
This is a test model because the previous attempt failed. So it turns out I also trained this incorrectly due to the dataset using wrongly formatted ShareGPT so after I trained it again "correctly" for another full epoch I noticed that so.. I will have to train it yet again.. but at least now it should be fixed. But this model is using the wrongly formatted dataset. The next model should be better.
Prompt format is: ChatML
LoRA: mpasila/Viking-SlimSonnet-v0.2-LoRA-7B
Trained with regular LoRA (not quantized/QLoRA) and LoRA rank was 128 and Alpha set to 32. Trained for 5000 steps (0.11 epoch).
Uploaded model
- Developed by: mpasila
- License: apache-2.0
- Finetuned from model : LumiOpen/Viking-7B
This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Languages
- en, fi, sv, no, da, is
- Created
- Aug 25, 2024
- Updated
- Aug 28, 2024