Downloads · 30 days
172
0% of all-time downloads
facebook-opt-125m-qcqa-ub-6-best-for-KV-cache
xformAI/facebook-opt-125m-qcqa-ub-6-best-for-KV-cache
facebook-opt-125m-qcqa-ub-6-best-for-KV-cache is a text generation model from xformAI. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as mit.
This is a QCQA version of the original model facebook/opt-125m. In this version, the original MHA architecture is preserved but instead of having a single K/V head, different K/V heads corresponding to the same group…
All-time downloads
37.4K
Public
Repo size
1 GB
Likes
15
Public
Repo makeup
Click a slice to open those files.
.bin501 MB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- mit
- Model type
- opt
- Access
- Public
- Created
- Jan 23, 2024
- Updated
- Jan 23, 2024
- SHA
- 9bfbe90e
Try a prompt
Model card
From the Hugging Face model README
This is a QCQA version of the original model facebook/opt-125m. In this version, the original MHA architecture is preserved but instead of having a single K/V head, different K/V heads corresponding to the same group have the same mean-pooled K or V values. It has upto 6 groups of KV heads per layer instead of original 12 KV heads in the MHA implementation. This implementation is supposed to more efficient than corresponding GQA one.
- Task
- Text Generation
- Library
- transformers
- Type
- opt
- License
- mit
- Languages
- en
- Created
- Jan 23, 2024
- Updated
- Jan 23, 2024