Aeshma-12B-imat-GGUF

About

This is a TIES merger made and quanted in the shadows of some of the more popular releases. I'm not sure it does anything better than anyone else, but when I started learning, I said I would add more quants for a model I thought was good enough to deserve it.
I may be biased, but I think it's good, and I've used it quite frequently under its formal test name Stage8c-12B

I'm going to leave this here along with the bf16 and imatrix.dat so that people can make their own quants if they want. In the future I'll re-visit this if I don't get sidetracked learning new stuff, since I've spent far too long fine tuning a soup recipe.

Usage

If you are unsure how to use GGUF files See: TheBloke's READMEs

Provided Quants

(sorted by file size, IQ-quants are often preferable over similar sized non-IQ quants)

Here is a graph by ikawrakow comparing some lower-quality quant types (as borrowed from mradermacher, along with some of my layout):

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: anthracite-org/magnum-v4-12b
    parameters:
      density: 0.82
      weight: 0.4
  - model: MarinaraSpaghetti/NemoMix-Unleashed-12B
    parameters:
      density: 0.69
      weight: 0.25
  - model: TheDrummer/Rocinante-12B-v1.1
    parameters:
      density: 0.69
      weight: 0.35
merge_method: ties
base_model: nbeerbower/mistral-nemo-bophades-12B
parameters:
  normalize: true
  int8_mask: true
dtype: bfloat16