mera-mix-4x7B-GGUF

New: mera-mix-4x7B GGUF

This is a repo for GGUF quants of mera-mix-4x7B. Currently it holds the FP16 and Q8_0 items only.

Original: Model mera-mix-4x7B

This is a mixture of experts (MoE) model that is half as large (4 experts instead of 8) as the Mixtral-8x7B while been comparable to it across different benchmarks. You can use it as a drop in replacement for your Mixtral-8x7B and get much faster inference.

mera-mix-4x7B achieves 76.37 on the openLLM eval v/s 72.7 by Mixtral-8x7B (as shown here).

You can try the model with the Mera Mixture Chat.

<!-- ## OpenLLM Eval | Model | ARC |HellaSwag|MMLU |TruthfulQA|Winogrande|GSM8K|Average| |-------------------------------------------------------------|----:|--------:|----:|---------:|---------:|----:|------:| |[mera-mix-4x7B](https://huggingface.co/meraGPT/mera-mix-4x7B)|72.01| 88.82|63.67| 77.45| 84.61|71.65| 76.37| Raw eval results are available at this [gist](https://gist.github.com/codelion/78f88333230801c9bbaa6fc22078d820) -->

Open LLM Leaderboard Evaluation Results

Detailed results can be found here