Miqu-MS-70B-3.2bpw-h6-exl2

<br/><br/> 3.2bpw/h6 exl2 quantization of Undi95/Miqu-MS-70B using default exllamav2 calibration dataset.

Fits in 32GB VRAM with 32k+ context (Q4cache)

---

ORIGINAL CARD:

Miqu-MS-70B

This is a merge of pre-trained language models created using mergekit.

The new MODEL STOCK merge method was used, see below for more information!

Feedback on this model is greatly appreciated! I hope this new merge method will be able to fill some hole Miqu have.

Others quant

• EXL2 (5.0 bpw) by lucyknada - measurement.json
• Static GGUF by mradermacher
• iMatrix GGUF by mradermacher - imatrix.dat

Thank you all!

Prompt format

Since it was made with model using different prompt format, the following should work.

Alpaca

### Instruction:
{system prompt}

### Input:
{prompt}

### Response:
{output}

Mistral

[INST] {prompt} [/INST] 

Vicuna

SYSTEM: <ANY SYSTEM CONTEXT>
USER: 
ASSISTANT:

Merge Details

Merge Method

This model was merged using the Model Stock merge method using 152334H/miqu-1-70b-sf as a base.

Models Merged

The following models were included in the merge:

• migtissera/Tess-70B-v1.6
• NeverSleep/MiquMaid-v2-70B
• sophosympatheia/Midnight-Miqu-70B-v1.0

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: NeverSleep/MiquMaid-v2-70B
  - model: sophosympatheia/Midnight-Miqu-70B-v1.0
  - model: migtissera/Tess-70B-v1.6
  - model: 152334H/miqu-1-70b-sf
merge_method: model_stock
base_model: 152334H/miqu-1-70b-sf
dtype: bfloat16

Support

If you want to support me, you can here.