Downloads · 30 days
20
4% of all-time downloads
c4ai-command-r-plus-GPTQ-ERQ
openerotica/c4ai-command-r-plus-GPTQ-ERQ
c4ai-command-r-plus-GPTQ-ERQ is a text generation model from openerotica. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as cc-by-nc-4.0.
c4ai-command-r-plus, quantized to GPTQ with these parameters:
All-time downloads
470
Public
Repo size
64.9 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.part010.7 GB · 17%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- cc-by-nc-4.0
- Model type
- cohere
- Access
- Public
- Created
- Jul 6, 2024
- Updated
- Jul 14, 2024
- SHA
- c7a413d9
Try a prompt
Model card
From the Hugging Face model README
c4ai-command-r-plus, quantized to GPTQ with these parameters:
python3 quant.py aCohereForAI/c4ai-command-r-plus /workspace/commandplus custom --bits 4 --group_size 128 --desc_act 1 --damp 0.1 --dtype float16 --seqlen 16384 --num_samples 256 --cache_examples 0 --trust_remote_code
The dataset used was openerotica/erotiquant2. I have included a script reconstitute.py to merge the files into one. Depending on the backend you might need to delete the index file after the files have been merged. I'll try to do this all in a better way once I work after I test out how marlin stacks up to exl2 for this model.
- Task
- Text Generation
- Library
- transformers
- Type
- cohere
- License
- cc-by-nc-4.0
- Created
- Jul 6, 2024
- Updated
- Jul 14, 2024