Downloads · 30 days
73
1% of all-time downloads
NEBULA-XB-v1.0_SFT_2_epoch
TeeZee/NEBULA-XB-v1.0_SFT_2_epoch
NEBULA-XB-v1.0_SFT_2_epoch is a text generation model from TeeZee. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Experiment, can DUS be taken one or more steps further?
All-time downloads
7.5K
Public
Parameters
23.8B
47.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors47.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- llama
- Access
- Public
- Created
- Apr 6, 2024
- Updated
- Apr 8, 2024
- SHA
- 5601ddb8
Try a prompt
Datasets
Model card
From the Hugging Face model README
TeeZee/NEBULA-XB-v1.0_SFT_2_epoch
Experiment, can DUS be taken one or more steps further?
Technical notes:
- pretrained model NEBULA-XB-v1.0 finetuned on 30k entries from Merge_Glue dataset
- 18 layers removed from both models of finetuned GALAXY-XB-v03
- model has 108 layers (((48-12)*2)-18)*2 = 108
- second step in scaling DUS procedure
To evaluate
- model performance after merge, should be a little lover that GALAXY finetuned on 50k of slimorca
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 58.02 |
| AI2 Reasoning Challenge (25-Shot) | 63.05 |
| HellaSwag (10-Shot) | 85.07 |
| MMLU (5-Shot) | 65.41 |
| TruthfulQA (0-shot) | 52.06 |
| Winogrande (5-shot) | 82.24 |
| GSM8k (5-shot) | 0.30 |
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- apache-2.0
- Created
- Apr 6, 2024
- Updated
- Apr 8, 2024