Downloads · 30 days
455
9% of all-time downloads
llama-3.1-instruct-bellman-8b-swedish
neph1/llama-3.1-instruct-bellman-8b-swedish
llama-3.1-instruct-bellman-8b-swedish is a machine learning model from neph1. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as llama3.1.
This version of bellman is finetuned from llama-3.1-instruct-8b. It's finetuned for prompt question answering, based on a dataset created from Swedish wikipedia, with a lot of Sweden-centric questions. New from previo…
All-time downloads
4.9K
Public
Repo size
35 GB
Likes
4
Public
Repo makeup
Click a slice to open those files.
.gguf34.4 GB · 98%
At a glance
- Library
- transformers
- License
- llama3.1
- Access
- Public
- Created
- Jul 24, 2024
- Updated
- Jul 26, 2024
- SHA
- 3d043646
Model card
From the Hugging Face model README
Model Card for Bellman
This version of bellman is finetuned from llama-3.1-instruct-8b. It's finetuned for prompt question answering, based on a dataset created from Swedish wikipedia, with a lot of Sweden-centric questions. New from previous versions is questions from a translated code-feedback dataset, as well as a number of stories. It's not great at generating stories, but better than previosly.
Try out the Q8 version here: https://huggingface.co/spaces/neph1/bellman (cpu)
Model Details
Training run on 240724:
Step Training Loss Validation Loss<br> 25 1.352200 1.034565<br> 50 1.033600 1.009348<br> 75 1.022400 0.996665<br> 100 1.002900 0.988050<br> 125 1.014600 0.981633<br> 150 1.006300 0.975584<br> 175 0.988800 0.970966<br> 200 0.985300 0.967037<br> 225 0.992400 0.964120<br> 250 0.950000 0.962472<br> 275 0.931000 0.960848<br> 300 0.932000 0.958946 <-- picked checkpoint <br>
Training Parameters
per_device_train_batch_size = 4,<br> gradient_accumulation_steps = 16,<br> num_train_epochs=3,<br> warmup_steps = 5,<br> learning_rate = 3e-5,<br> logging_steps = 25,<br> optim = "adamw_8bit",<br> weight_decay = 0.01,<br> lr_scheduler_type = "linear",<br> seed = 3407,<br> per_device_eval_batch_size = 2,<br> eval_strategy="steps",<br> eval_accumulation_steps = 32,<br> eval_steps = 25,<br> eval_delay = 0,<br> save_strategy="steps",<br> save_steps=50,<br>
Model Description
- Developed by: Me
- Funded by: Me
- Model type: Instruct
- Language(s) (NLP): Swedish
- License: llama-3.1
- Finetuned from model: Llama3.1 Instruct 8b
Model Card Contact
- Library
- transformers
- License
- llama3.1
- Languages
- sv
- Created
- Jul 24, 2024
- Updated
- Jul 26, 2024