Downloads · 30 days
25
8% of all-time downloads
Xwinter-120b
llmixer/Xwinter-120b
Xwinter-120b is a text generation model from llmixer. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
<img src="https://cdn-uploads.huggingface.co/production/uploads/65a6db055c58475cf9e6def1/gs7KMVn4GJuksiuJIm.png" width=600
All-time downloads
305
Public
Parameters
118B
235 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors235 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- llama2
- Model type
- llama
- Access
- Public
- Created
- Jan 17, 2024
- Updated
- Jan 17, 2024
- SHA
- 390acf01
Try a prompt
Model card
From the Hugging Face model README
Xwinter 120B
<img src="https://cdn-uploads.huggingface.co/production/uploads/65a6db055c58475cf9e6def1/gs7KMVn4__G_JuksiuJIm.png" width=600>A Goliath-120b style frankenmerge of Xwin-LM-70b-v0.1 and WinterGoddess-1.4x-70b. Meant as a slight update to Goliath by using the successor model to Euryale.
There's a similar merge called WinterGoliath-123b by @ChuckMcSneed.
Prompting Format
Vicuna and Alpaca.
Merge process
The models used in the merge are Xwin-LM-70b-v0.1 and WinterGoddess-1.4x-70b.
The layer mix:
- range 0, 16
Xwin
- range 8, 24
WinterGoddess
- range 17, 32
Xwin
- range 25, 40
WinterGoddess
- range 33, 48
Xwin
- range 41, 56
WinterGoddess
- range 49, 64
Xwin
- range 57, 72
WinterGoddess
- range 65, 80
Xwin
Acknowledgements
@Xwin-LM For creating Xwin
@Sao10K For creating WinterGoddess
@alpindale For creating the original Goliath
@chargoddard For developing mergekit.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Languages
- en
- Created
- Jan 17, 2024
- Updated
- Jan 17, 2024