Downloads · 30 days
18
27% of all-time downloads
StableMedZephyr-Merged-3b
Heng666/StableMedZephyr-Merged-3b
StableMedZephyr-Merged-3b is a text generation model from Heng666. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as other.
stabilityai/stablelm-zephyr-3b, StableMed-3b merged with a new, experimental implementation of "dare ties" via mergekit. See:
All-time downloads
66
Public
Parameters
2.8B
5.6 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors5.6 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- other
- Model type
- stablelm_epoch
- Access
- Public
- Created
- Dec 28, 2023
- Updated
- Dec 28, 2023
- SHA
- 7b42a3a9
Try a prompt
Model card
From the Hugging Face model README
FOR EXPERIMENT
Description
stabilityai/stablelm-zephyr-3b, StableMed-3b merged with a new, experimental implementation of "dare ties" via mergekit. See:
Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch
Usage
StableLM Zephyr 3B uses the following instruction format:
<|user|>
List 3 synonyms for the word "tiny"<|endoftext|>
<|assistant|>
1. Dwarf
2. Little
3. Petite<|endoftext|>
Testing Notes
Merged in mergekit with the following config, and the tokenizer from chargoddard's Yi-Llama:
models:
- model: stabilityai/stablelm-zephyr-3b
# no parameters necessary for base model
- model: cxllin/StableMed-3b
parameters:
weight: 0.08
density: 0.5
merge_method: dare_ties
base_model: stabilityai/stablelm-zephyr-3b
parameters:
int8_mask: true
dtype: bfloat16
Model Details
- Task
- Text Generation
- Library
- transformers
- Type
- stablelm_epoch
- License
- other
- Languages
- en
- Created
- Dec 28, 2023
- Updated
- Dec 28, 2023