chargoddard

Mistral-7B-MHA

chargoddard/Mistral-7B-MHA

Mistral-7B-MHA is a text generation model from chargoddard. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.

Mistral-7B-v0.1 with the kv heads duplicated to not use GQA. Don't use this directly - it'll give you the same results, just slower. Meant for merging experiments.