Downloads · 30 days
16
19% of all-time downloads
gpt-sw3-6.7b-v2-instruct-slimorcasv
skvarre/gpt-sw3-6.7b-v2-instruct-slimorcasv
gpt-sw3-6.7b-v2-instruct-slimorcasv is a text generation model from skvarre. Use it when you need the model to write or continue text. It is set up for transformers.
Finetune of gpt-sw3-6.7b-v2 using LoRA with 4-bit quantization. Adapters merged with base model, however with precision tensors.
All-time downloads
86
Public
Parameters
6.7B
13.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors13.4 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- gpt2
- Access
- Public
- Created
- May 7, 2024
- Updated
- May 13, 2024
- SHA
- 77dd25f7
Try a prompt
Base models
Datasets
Model card
From the Hugging Face model README
Model Details
Finetune of gpt-sw3-6.7b-v2 using LoRA with 4-bit quantization. Adapters merged with base model, however with bfloat16 precision tensors.
Usage:
This is a finetune experiment. How to use will be provided later.
Evaluation
ScandEval benchmarks:
| Dataset | Performance (Metric 1 / Metric 2) |
|---|---|
| swerec | 74.95 ± 1.17 / 61.38 ± 1.37 |
| suc3 | 30.75 ± 4.11 / 25.69 ± 4.83 |
| scala-sv | 8.96 ± 2.09 / 51.50 ± 2.94 |
| scandiqa-sv | 50.71 ± 0.99 / 56.76 ± 0.89 |
| swedn | 64.37 ± 0.72 / 18.25 ± 0.29 |
| mmlu-sv | 5.45 ± 0.91 / 28.14 ± 0.82 |
| hellaswag-sv | 27.95 ± 0.73 / 4.19 ± 0.94 |
| speed | 5322.20 ± 1132.75 / 1280.06 ± 408.08 |
- Task
- Text Generation
- Library
- transformers
- Type
- gpt2
- Created
- May 7, 2024
- Updated
- May 13, 2024