YiSM-34B-0rn

<img src=https://huggingface.co/altomek/YiSM-34B-0rn/resolve/main/YiSM.png> <a href="https://www.youtube.com/watch?v=a9dNpk9G5h0" title="P.T. Adamczyk - Never Looking Back | Cyberpunk 2077: Phantom Liberty (Original Score)" target="_blank">intro music...</a>

YiSM-34B-0rn

This is Yi Self Merged. I wanted model that will follow most instuctions yet preserve its base model nature.

Ingridients

• Yi-1.5-34B-Chat

• Yi-1.5-34B

Settings

I use max_seq_len 8K with alpha_value 2.65.

SillyTavern presets:

{
    "temp": 0.1,
    "temperature_last": true,
    "top_p": 1,
    "top_k": 0,
    "top_a": 0,
    "tfs": 1,
    "epsilon_cutoff": 0,
    "eta_cutoff": 0,
    "typical_p": 1,
    "min_p": 0,
    "rep_pen": 1.08,
    "rep_pen_range": 0,
    "no_repeat_ngram_size": 0,
    "penalty_alpha": 0,
    "num_beams": 1,
    "length_penalty": 1,
    "min_length": 0,
    "encoder_rep_pen": 1,
    "freq_pen": 0.01,
    "presence_pen": 0,
    "do_sample": true,
    "early_stopping": false,
    "add_bos_token": true,
    "truncation_length": 2048,
    "ban_eos_token": false,
    "skip_special_tokens": true,
    "streaming": true,
    "mirostat_mode": 0,
    "mirostat_tau": 5,
    "mirostat_eta": 0.1,
    "guidance_scale": 1,
    "negative_prompt": "",
    "grammar_string": "",
    "banned_tokens": "",
    "ignore_eos_token_aphrodite": false,
    "spaces_between_special_tokens_aphrodite": true,
    "sampler_order": [
        6,
        0,
        1,
        3,
        4,
        2,
        5
    ],
    "logit_bias": [],
    "n": 1,
    "rep_pen_size": 0,
    "genamt": 2048,
    "max_length": 8192
}

Terms and Conditions of Use

The following table outlines the primary characteristics and intended uses of my YiSM-34B-0rn models:

Please remember that all YiSM-34B-0rn models operate under the apache-2.0 license, so familiarize yourself with its terms and conditions before employing their content.

Quants

• GGUF
• 8bpw
• 6.5bpw
• 4.65bpw
• 4bpw
• 3.2bpw -> Fits in 16GB VRAM but not recomended. Performance is significantly degraded in lower quants.
• measurements --> ExLlamav2 measurments

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

5th in 34B size range excluding "Private or deleted" or 8th with all models included as of 2024-06-10 ;P <img src=https://huggingface.co/altomek/YiSM-34B-0rn/resolve/main/5thIn34B.png>

Open LLM Leaderboard Evaluation Results

Detailed results can be found here