Fimbulvetr-11B-v2-4.35bpw-exl2

Cute girl to catch your attention.

EXL2 4.35bpw quant of (https://huggingface.co/Sao10K/Fimbulvetr-11B-v2)

Same goal as my last quants. being able to fit 16k context within 8GB of ram with as much quality as possible. This time with 11b!

Recommended to run with Tabbyapi, ooba could cause some issues with model loading and running.

Tested using RTX 4060 8GB Laptop, 16K context with 4bit cache.

https://huggingface.co/Sao10K/Fimbulvetr-11B-v2-GGUF <------ GGUF

Fimbulvetr-v2 - A Solar-Based Model

Prompt Formats - Alpaca or Vicuna. Either one works fine. Recommended SillyTavern Presets - Universal Light

Alpaca:

### Instruction:
<Prompt>
### Input:
<Insert Context Here>
### Response:

Vicuna:

System: <Prompt>

User: <Input>

Assistant:

---

Changelogs:

25/2 - repo renamed to remove test, model card redone. Model's officially out. <br>15/2 - Heavy testing complete. Good feedback.

---

<details><summary>Rant - Kept For Historical Reasons</summary>

Ramble to meet minimum length requirements:

Tbh i wonder if this shit is even worth doing. Like im just some broke guy lmao I've spent so much. And for what? I guess creds. Feels good when a model gets good feedback, but it seems like im invisible sometimes. I should be probably advertising myself and my models on other places but I rarely have the time to. Probably just internal jealousy sparking up here and now. Wahtever I guess.

Anyway cool EMT vocation I'm doing is cool except it pays peanuts, damn bruh 1.1k per month lmao. Government to broke to pay for shit. Pays the bills I suppose.

Anyway cool beans, I'm either going to continue the Solar Train or go to Mixtral / Yi when I get paid.

You still here?

</details><br>