Downloads · 30 days
4
5% of all-time downloads
invisietch/HauntingEcho-v0.1-rc1-18B-GGUF
HauntingEcho-v0.1-rc1-18B-GGUF is a machine learning model from invisietch. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as llama3.1.
<div align="center" <b style="font-size: 36px;"HauntingEcho-v0.1-rc1-18B</b
Downloads · 30 days
4
5% of all-time downloads
All-time downloads
81
Public
Repo size
34.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf34.9 GB · 100%
From the Hugging Face model README
This is a release candidate model. That means it has not been tested as extensively as my full releases and as such you should expect various issues. The primary purpose of uploading this model is to get feedback.
HauntingEcho is a passthrough merge which attempts to merge together RP-capable Llama 3 8B models with the smartest Llama 3.1 8B models.
The primary aim of this merge is to create a Llama 3-based roleplay model which combines the context capabilities of Llama 3.1 with the roleplay ability and reduced censorship of Llama 3 tunes. So far, we don't seem to have achieved longer context, this is still a 12-16k context model.
The secondary aim was to create a model size which runs well on consumer cards. Most people seem to run 16GB or 24GB cards, so an 18B seems like a perfect size for the Q8_0 to run in 24GB VRAM and the Q6_K to run in 16GB VRAM.
Please, please, please, give feedback on this merge. It is simply the result of a theory and minimally tested to ensure that it doesn't completely collapse. I welcome feedback in the community section here, or on Discord (name is the same as on here). I can be found on the Chub & SillyTavern Discord servers in the local LLM channels frequently too.
As this is a release candidate, I have only provided GGUFs.
This model is built on an abliterated base and as such is largely uncensored. It can generate explicit, disturbing or offensive responses. Use responsibly. I am not responsible for your use of this model.
I'm using these sampler settings:
These are by no means the perfect settings, feel free to experiment.
I'd recommend Llama-3 Instruct prompting format:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{output}<|eot_id|>
Some of the models included in the merge were trained on ChatML & Alpaca so you can try those. I have not tested them.
N/A.
The following models were used to create EtherealRainbow-v0.3-8B:
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 24]
model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
- sources:
- layer_range: [8,24]
model: invisietch/EtherealRainbow-v0.3-8B
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8,24]
model: akjindal53244/Llama-3.1-Storm-8B
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8,24]
model: invisietch/EtherealRainbow-v0.3-8B
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [24, 32]
model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2