Downloads · 30 days
0
invisietch/Whalesong-v0.1-rc1-25B-TEST-GGUF
Whalesong-v0.1-rc1-25B-TEST-GGUF is a machine learning model from invisietch. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
<div align="center" <b style="font-size: 36px;"Whalesong-v0.1-rc1-25B-GGUF</b
Downloads · 30 days
0
Access
Public
Updated Oct 11, 2024
Repo size
17.8 GB
Likes
0
Public
Click a slice to open those files.
.1-rc1-25b-q5_k_m17.8 GB · 100%
From the Hugging Face model README
This is a release candidate model. That means it has not been tested as extensively as my full releases and as such you should expect various issues. The primary purpose of uploading this model is to get feedback.
Whalesong is a passthrough upscale using Atlantis 12B v0.1 along with Lyra 12B, StarDust & NemoMix to build a 25B model for roleplay and creative writing.
Please, please, please, give feedback on this merge. It is simply the result of a theory and minimally tested to ensure that it doesn't completely collapse. I welcome feedback in the community section here, or on Discord (name is the same as on here). I can be found on the Chub & SillyTavern Discord servers in the local LLM channels frequently too.
As this is a release candidate, I have only provided GGUFs.
This model is built on an abliterated base and as such is largely uncensored. It can generate explicit, disturbing or offensive responses. Use responsibly. I am not responsible for your use of this model.
I'm using these sampler settings with SillyTavern:
<details><summary>Copy this and save as a JSON then import to SillyTavern samplers:</summary> <tt> { "temp": 0.5, "temperature_last": true, "top_p": 0.9, "top_k": 0, "top_a": 0, "tfs": 1, "epsilon_cutoff": 0, "eta_cutoff": 0, "typical_p": 1, "min_p": 0.1, "rep_pen": 1.09, "rep_pen_range": 1536, "rep_pen_decay": 0, "rep_pen_slope": 1, "no_repeat_ngram_size": 0, "penalty_alpha": 0, "num_beams": 1, "length_penalty": 1, "min_length": 0, "encoder_rep_pen": 1, "freq_pen": 0.01, "presence_pen": 0, "skew": 0, "do_sample": true, "early_stopping": false, "dynatemp": true, "min_temp": 0.35, "max_temp": 0.65, "dynatemp_exponent": 1, "smoothing_factor": 0, "smoothing_curve": 1, "dry_allowed_length": 2, "dry_multiplier": 0, "dry_base": 1.75, "dry_sequence_breakers": "[\"\\n\", \":\", \"\\\"\", \"*\"]", "dry_penalty_last_n": 1536, "add_bos_token": true, "ban_eos_token": false, "skip_special_tokens": true, "mirostat_mode": 0, "mirostat_tau": 5, "mirostat_eta": 0.1, "guidance_scale": 1, "negative_prompt": "", "grammar_string": "", "json_schema": {}, "banned_tokens": "", "sampler_priority": [ "repetition_penalty", "presence_penalty", "frequency_penalty", "dry", "temperature", "dynamic_temperature", "quadratic_sampling", "top_k", "top_p", "typical_p", "epsilon_cutoff", "eta_cutoff", "tfs", "top_a", "min_p", "mirostat", "xtc", "encoder_repetition_penalty", "no_repeat_ngram" ], "samplers": [ "top_k", "tfs_z", "typical_p", "top_p", "min_p", "temperature" ], "ignore_eos_token": false, "spaces_between_special_tokens": true, "speculative_ngram": false, "sampler_order": [ 6, 0, 1, 3, 4, 2, 5 ], "logit_bias": [], "xtc_threshold": 0.1, "xtc_probability": 0.25, "rep_pen_size": 0, "genamt": 2048, "max_length": 16384 } </tt> </details>If you don't use SillyTavern, I'd recommend the following:
Note: This is a Nemo merge. If you use temperatures above 0.5-0.6, expect responses that aren't coherent and don't adhere to your character cards. I use dynamic temperature as it gives me a good midpoint between coherency and creativity without too much predictability.
I'd recommend Mistral V3-Tekken prompting format:
<s>[INST]User message[/INST]Bot message.</s>
N/A.
The following models were used to create Whalesong v0.1 release candidate 1:
dtype: bfloat16
merge_method: passthrough
slices:
- sources:
- layer_range: [0, 31]
model: invisietch/Atlantis-v0.1-12B
- sources:
- layer_range: [16, 32]
model: Sao10K/MN-12B-Lyra-v4
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [16, 32]
model: Luni/StarDust-12b-v2
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [16,32]
model: MarinaraSpaghetti/NemoMix-Unleashed-12B
parameters:
scale:
- filter: o_proj
value: 0.0
- filter: down_proj
value: 0.0
- value: 1.0
- sources:
- layer_range: [32, 40]
model: invisietch/Atlantis-v0.1-12B