Downloads · 30 days
14
7% of all-time downloads
zombyi-12b
reallad/zombyi-12b
zombyi-12b is a text generation model from reallad. Use it when you need the model to write or continue text. It is set up for transformers.
This outputs coherent words sometimes. It's the result of an attempt to merge Llama-2 and Yi-6B. Coherent words are a victory but it seems resistant to further fine tuning with QLORA and I'm not inclined to spend the…
All-time downloads
210
Public
Parameters
11.5B
22.9 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors22.9 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- Model type
- llama
- Access
- Public
- Created
- Apr 17, 2024
- Updated
- Apr 19, 2024
- SHA
- 46d47a9d
Try a prompt
Model card
From the Hugging Face model README
This outputs coherent words sometimes. It's the result of an attempt to merge Llama-2 and Yi-6B. Coherent words are a victory but it seems resistant to further fine tuning with QLORA and I'm not inclined to spend the GPU resources required for a full fine tune.
zombyi
This is a merge of pre-trained language models created using mergekit.
Merge Details
Merge Method
This model was merged using the passthrough merge method using as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
slices:
- sources:
- model: reallad/llamayialpaca
layer_range: [0,32]
- sources:
- model: reallad/lesslobollama2
layer_range: [0,32]
merge_method: passthrough
dtype: bfloat16
base_model: reallad/llamayialpaca
tokenizer_source: reallad/llamayialpaca
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- Created
- Apr 17, 2024
- Updated
- Apr 19, 2024