Downloads · 30 days
17
0% of all-time downloads
Codestral-22B-v0.1-GGUF
mukel/Codestral-22B-v0.1-GGUF
Codestral-22B-v0.1-GGUF is a machine learning model from mukel. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as other.
In the wild, Q80 quantizations are fine, but Q40 quantizations are rarely pure e.g. the output.weights tensor is quantized with Q6K, instead of Q40. A pure Q40 quantization can be generated from a high precision (F32,…
All-time downloads
3.6K
Public
Repo size
36.2 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.gguf36.2 GB · 100%
At a glance
- License
- other
- Access
- Public
- Created
- May 29, 2024
- Updated
- May 31, 2024
- SHA
- e23d0532
Model card
From the Hugging Face model README
Pure quantizations of Codestral-22B-v0.1 for mistral.java.
In the wild, Q8_0 quantizations are fine, but Q4_0 quantizations are rarely pure e.g. the output.weights tensor is quantized with Q6_K, instead of Q4_0.
A pure Q4_0 quantization can be generated from a high precision (F32, F16, BFLOAT16) .gguf source with the quantize utility from llama.cpp as follows:
./quantize --pure ./Codestral-22B-v0.1-F32.gguf ./Codestral-22B-v0.1-Q4_0.gguf Q4_0
Original model: https://huggingface.co/mistralai/Codestral-22B-v0.1
**Note that this model does not support a System prompt.
Codestrall-22B-v0.1 is trained on a diverse dataset of 80+ programming languages, including the most popular ones, such as Python, Java, C, C++, JavaScript, and Bash (more details in the Blogpost). The model can be queried:
- As instruct, for instance to answer any questions about a code snippet (write documentation, explain, factorize) or to generate code following specific indications
- As Fill in the Middle (FIM), to predict the middle tokens between a prefix and a suffix (very useful for software development add-ons like in VS Code)
- License
- other
- Languages
- code
- Created
- May 29, 2024
- Updated
- May 31, 2024