xformAI

opt-6.7b-ub-16-qcqa-best-for-KV-cache

xformAI/opt-6.7b-ub-16-qcqa-best-for-KV-cache

opt-6.7b-ub-16-qcqa-best-for-KV-cache is a machine learning model from xformAI. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as mit.

This is a QCQA version of the original model facebook/opt-125m. In this version, the original MHA architecture is preserved but instead of having a single K/V head, different K/V heads corresponding to the same group…