Downloads · 30 days
12
10% of all-time downloads
invisietch/RainbowsEdge-v0.1-rc1-18B-GGUF
RainbowsEdge-v0.1-rc1-18B-GGUF is a machine learning model from invisietch. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as llama3.
<div align="center" <b style="font-size: 36px;"RainbowsEdge-v0.1-rc1-18B-GGUF</b
Downloads · 30 days
12
10% of all-time downloads
All-time downloads
116
Public
Repo size
32.8 GB
Likes
0
Public
Click a slice to open those files.
.gguf32.8 GB · 100%
From the Hugging Face model README
<b style="font-size: 24px;color:#ffff00">Important note: this model has a broken config which prevents ROPE from working. Please use <a href="https://huggingface.co/invisietch/RainbowsEdge-v0.1-rc2-18B-GGUF">release candidate 2</a> instead. It is an identical model apart from fixing this issue, and supporting closer to 16k context with ROPE.</b>
<img src="https://huggingface.co/invisietch/RainbowsEdge-v0.1-rc1-18B-GGUF/resolve/main/header.png" style="width:60%"> </div>This is a release candidate model. That means it has not been tested as extensively as my full releases and as such you should expect various issues. The primary purpose of uploading this model is to get feedback.
Rainbow's Edge is a passthrough upscale of two highly capable Llama 3 8B roleplay models, with a goal to create a very uncensored and very capable release model which matches a lot of GPUs.
The Q8_0 runs well on a 24GB GPU.
The Q5_K_M runs well on a 16GB GPU.
Due to this being a Llama 3 (not Llama 3.1) merge, context sizes above 8192 are not supported or recommended. Catastrophic collapse is the expected behavior beyond this context length.
Please, please, please, give feedback on this merge. It is simply the result of a theory and minimally tested to ensure that it doesn't completely collapse. I welcome feedback in the community section here, or on Discord (name is the same as on here). I can be found on the Chub & SillyTavern Discord servers in the local LLM channels frequently too.
As this is a release candidate, I have only provided GGUFs.
This model is built on an abliterated base and as such is largely uncensored. It can generate explicit, disturbing or offensive responses. Use responsibly. I am not responsible for your use of this model.
I'm using these sampler settings:
These are by no means the perfect settings, feel free to experiment.
I'd recommend Llama-3 Instruct prompting format:
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
{input}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
{output}<|eot_id|>
Some of the models included in the merge were trained on ChatML & Alpaca so you can try those. I have not tested them.
N/A.
The following models were used to create RainbowsEdge v0.1 RC1:
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 24]
model: invisietch/EtherealRainbow-v0.3-8B
- sources:
- layer_range: [8,24]
model: SicariusSicariiStuff/Dusk_Rainbow
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8,24]
model: invisietch/EtherealRainbow-v0.3-8B
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [8,24]
model: SicariusSicariiStuff/Dusk_Rainbow
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [24, 32]
model: invisietch/EtherealRainbow-v0.3-8B