Downloads · 30 days
17
8% of all-time downloads
CodeLlama-34b-hf-4bit
cmarkea/CodeLlama-34b-hf-4bit
CodeLlama-34b-hf-4bit is a text generation model from cmarkea. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as llama2.
Converted version of CodeLlama-34b to 4-bit using bitsandbytes. For more information about the model, refer to the model's page.
All-time downloads
207
Public
Parameters
34.8B
18.2 GB on disk
Likes
1
Public
Repo makeup
Click a slice to open those files.
.safetensors18.2 GB · 100%
Parameter types
How the weights are stored.
U834.3B · 98%
Try a prompt
Model card
From the Hugging Face model README
Converted version of CodeLlama-34b to 4-bit using bitsandbytes. For more information about the model, refer to the model's page.
Impact on performance
In the following figure, we can see the impact on the performance of a set of models relative to the required RAM space. It is noticeable that the quantized models have equivalent performance while providing a significant gain in RAM usage.
- Task
- Text Generation
- Library
- transformers
- Type
- llama
- License
- llama2
- Created
- Aug 9, 2024
- Updated
- Aug 14, 2024