Downloads · 30 days
3
3% of all-time downloads
invisietch/HauntingEcho-v0.1-rc2-18B-GGUF
HauntingEcho-v0.1-rc2-18B-GGUF is a machine learning model from invisietch. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as llama3.1.
<div align="center" <b style="font-size: 36px;"HauntingEcho-v0.1-rc2-18B</b
Downloads · 30 days
3
3% of all-time downloads
All-time downloads
112
Public
Repo size
36.2 GB
Likes
0
Public
Click a slice to open those files.
.gguf36.2 GB · 100%
From the Hugging Face model README
This is a release candidate model. That means it has not been tested as extensively as my full releases and as such you should expect various issues. The primary purpose of uploading this model is to get feedback.
HauntingEcho is a passthrough merge which attempts to merge together RP-capable Llama 3 8B models with the smartest Llama 3.1 8B models.
The original aim of this merge was to use Llama 3.1 8B as a base to create a 32k context RP model with the popular feel of L3 8B RP models like EtherealRainbow. It seems like 32k context is not achievable, but 14-16k is, and that's a significant improvement over L3 8B.
The new aim of this merge strategy is to hit 16k context, to give people a slightly smarter alternative to Nemo with a Llama 3 feel and the freedom to play with the temperature that Llama 3 offers. 18B has been chosen as a very deliberate size, to offer a model that maximizes parameter count & quantization quality for popular GPU sizes: 24GB GPUs can run the 8bpw model, and 16GB GPUs can run the 5bpw model easily.
Please, please, please, give feedback on this merge. It is simply the result of a theory and minimally tested to ensure that it doesn't completely collapse. I welcome feedback in the community section here, or on Discord (name is the same as on here). I can be found on the Chub & SillyTavern Discord servers in the local LLM channels frequently too.
As this is a release candidate, I have only provided GGUFs.
This model is built on an abliterated base and as such is largely uncensored. It can generate explicit, disturbing or offensive responses. Use responsibly. I am not responsible for your use of this model.
I'm using these sampler settings:
These are by no means the perfect settings, feel free to experiment.
I'd recommend Llama-3 Instruct prompting format:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{output}<|eot_id|>
Some of the models included in the merge were trained on ChatML & Alpaca so you can try those. I have not tested them.
N/A.
The following models were used to create EtherealRainbow-v0.3-8B:
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 24]
model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
- sources:
- layer_range: [8,24]
model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8,24]
model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8,24]
model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [24, 32]
model: Orenguteng/Llama-3.1-8B-Lexi-Uncensored-V2
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 24]
model: invisietch/EtherealRainbow-v0.3-8B
- sources:
- layer_range: [8,24]
model: invisietch/EtherealRainbow-v0.3-8B
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8,24]
model: invisietch/EtherealRainbow-v0.3-8B
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8,24]
model: invisietch/EtherealRainbow-v0.3-8B
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [24, 32]
model: invisietch/EtherealRainbow-v0.3-8B
dtype: bfloat16
models:
- model: /mnt/models/L3.1-Uncensored-extended
- model: /mnt/models/ER03-extended
merge_method: slerp
base_model: /mnt/models/L3.1-Uncensored-extended
parameters:
t:
- value: [0, 0, 0.25, 0.35, 0.5, 0.75, 0.5, 0.35, 0.25, 0, 0]
embed_slerp: true