Downloads · 30 days
171
4% of all-time downloads
danube2-upscale-1.7
Lambent/danube2-upscale-1.7
danube2-upscale-1.7 is a text generation model from Lambent. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This is a merge of pre-trained language models created using mergekit.
All-time downloads
4.2K
Public
Parameters
2.2B
4.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Apr 21, 2024
- Updated
- Apr 21, 2024
- SHA
- b0aa3dac
Try a prompt
Base models
Model card
From the Hugging Face model README
eq90parsedanube
This is a merge of pre-trained language models created using mergekit.
First one that's shown promising capability improvement over the base model h2o-danube2-1.8b-base.
Training methodology ... is a bit of a mess, trying out different things. I'm adding the datasets used at any point, but I don't think replicating the recipe is doable or sensible.
Original upscale at Lambent/danube2-upscale-1, duplicating layers 16-21. Various training methods attempted to repair. Linear merge is of the 4 that were at least 90% parseable by the EQ-Bench benchmark.
| Model | AGIEval | GPT4All | TruthfulQA | Bigbench | Average |
|---|---|---|---|---|---|
| danube2-upscale-1.7 | 27.97 | 62.16 | 42.2 | 32.2 | 41.13 |
| Model | EQ-Bench | Average |
|---|---|---|
| danube2-upscale-1.7 | 15.52 | 15.52 |
EQ-Bench
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| eq_bench | 2.1 | eqbench,none | 15.52 | ||
| eqbench_stderr,none | 2.77 | ||||
| percent_parseable,none | 100 | ||||
| percent_parseable_stderr,none | 0 | ||||
| alias | eq_bench |
Average: 15.52%
Average score: 15.52%
Merge Details
Merge Method
This model was merged using the linear merge method.
Models Merged
The following models were included in the merge:
- Lambent/danube2-upscale-1.53lisa
- Lambent/danube2-upscale-1.51galore
- Lambent/danube2-upscale-1.531qlora
- Lambent/danube2-upscale-1.51qlora
Configuration
The following YAML configuration was used to produce this model:
models:
- model: Lambent/danube2-upscale-1.531qlora
parameters:
weight: 1.0
- model: Lambent/danube2-upscale-1.53lisa
parameters:
weight: 1.0
- model: Lambent/danube2-upscale-1.51galore
parameters:
weight: 1.0
- model: Lambent/danube2-upscale-1.51qlora
parameters:
weight: 1.0
merge_method: linear
dtype: float16
- Task
- Text Generation
- Library
- transformers
- Type
- mistral
- License
- apache-2.0
- Created
- Apr 21, 2024
- Updated
- Apr 21, 2024