Downloads · 30 days
11
30% of all-time downloads
NemoMix-12B-DellaV1a
jsfs11/NemoMix-12B-DellaV1a
NemoMix-12B-DellaV1a is a machine learning model from jsfs11. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
NemoMix-12B-DellaV1a is an experimental merge of the following models using the DELLA method with mergekit: BeaverAI/mistral-doryV2-12b NeverSleep/Lumimaid-v0.2-12B
All-time downloads
37
Public
Parameters
12.2B
24.5 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors24.5 GB · 100%
At a glance
- License
- apache-2.0
- Model type
- mistral
- Access
- Public
- Created
- Aug 2, 2024
- Updated
- Aug 2, 2024
- SHA
- da7f55eb
Model card
From the Hugging Face model README
NemoMix-12B-DellaV1a
NemoMix-12B-DellaV1a is an experimental merge of the following models using the DELLA method with mergekit:
- BeaverAI/mistral-doryV2-12b
- NeverSleep/Lumimaid-v0.2-12B
- intervitens/mini-magnum-12b-v1.1
- grimjim/mistralai-Mistral-Nemo-Instruct-2407
EDIT: There seem to be tokenizer issues. I would probably have to merge with the base model instead of instruct I'm guessing. Don't bother.
🧩 Configuration
models:
- model: BeaverAI/mistral-doryV2-12b
parameters:
weight: 0.20
density: 0.42
- model: NeverSleep/Lumimaid-v0.2-12B
parameters:
weight: 0.22
density: 0.54
- model: intervitens/mini-magnum-12b-v1.1
parameters:
weight: 0.24
density: 0.66
- model: grimjim/mistralai-Mistral-Nemo-Instruct-2407
parameters:
weight: 0.34
density: 0.78
merge_method: della
base_model: grimjim/mistralai-Mistral-Nemo-Instruct-2407
parameters:
int8_mask: true
epsilon: 0.1
lambda: 1.0
density: 0.7
dtype: bfloat16
- Type
- mistral
- License
- apache-2.0
- Created
- Aug 2, 2024
- Updated
- Aug 2, 2024