Downloads · 30 days
149
7% of all-time downloads
TroyDoesAI_-_Codestral-21B-Pruned-gguf
RichardErkhov/TroyDoesAI_-_Codestral-21B-Pruned-gguf
TroyDoesAI_-_Codestral-21B-Pruned-gguf is a machine learning model from RichardErkhov. Use it for the machine learning task on the model card, and read the license before you ship it in a product.
All-time downloads
2.2K
Public
Repo size
243 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf281 GB · 100%
At a glance
- Access
- Public
- Created
- Oct 20, 2024
- Updated
- Oct 21, 2024
- SHA
- dbe7dd80
Model card
From the Hugging Face model README
Quantization made by Richard Erkhov.
Codestral-21B-Pruned - GGUF
- Model creator: https://huggingface.co/TroyDoesAI/
- Original model: https://huggingface.co/TroyDoesAI/Codestral-21B-Pruned/
| Name | Quant method | Size |
|---|---|---|
| Codestral-21B-Pruned.Q2_K.gguf | Q2_K | 7.44GB |
| Codestral-21B-Pruned.IQ3_XS.gguf | IQ3_XS | 8.24GB |
| Codestral-21B-Pruned.IQ3_S.gguf | IQ3_S | 8.71GB |
| Codestral-21B-Pruned.Q3_K_S.gguf | Q3_K_S | 8.67GB |
| Codestral-21B-Pruned.IQ3_M.gguf | IQ3_M | 9.04GB |
| Codestral-21B-Pruned.Q3_K.gguf | Q3_K | 9.67GB |
| Codestral-21B-Pruned.Q3_K_M.gguf | Q3_K_M | 9.67GB |
| Codestral-21B-Pruned.Q3_K_L.gguf | Q3_K_L | 10.54GB |
| Codestral-21B-Pruned.IQ4_XS.gguf | IQ4_XS | 10.82GB |
| Codestral-21B-Pruned.Q4_0.gguf | Q4_0 | 11.3GB |
| Codestral-21B-Pruned.IQ4_NL.gguf | IQ4_NL | 11.41GB |
| Codestral-21B-Pruned.Q4_K_S.gguf | Q4_K_S | 11.37GB |
| Codestral-21B-Pruned.Q4_K.gguf | Q4_K | 11.96GB |
| Codestral-21B-Pruned.Q4_K_M.gguf | Q4_K_M | 11.96GB |
| Codestral-21B-Pruned.Q4_1.gguf | Q4_1 | 12.53GB |
| Codestral-21B-Pruned.Q5_0.gguf | Q5_0 | 13.77GB |
| Codestral-21B-Pruned.Q5_K_S.gguf | Q5_K_S | 13.77GB |
| Codestral-21B-Pruned.Q5_K.gguf | Q5_K | 14.12GB |
| Codestral-21B-Pruned.Q5_K_M.gguf | Q5_K_M | 14.12GB |
| Codestral-21B-Pruned.Q5_1.gguf | Q5_1 | 15.01GB |
| Codestral-21B-Pruned.Q6_K.gguf | Q6_K | 16.4GB |
| Codestral-21B-Pruned.Q8_0.gguf | Q8_0 | 21.24GB |
Original model description:
license: apache-2.0 language:
- en metrics:
- accuracy
- code_eval library_name: transformers pipeline_tag: text-generation tags:
- rag
- context obedient
- TroyDoesAI
- Mermaid
- Flow
- Diagram
- Sequence
- Map
- Context
- Accurate
- Summarization
- Story
- Code
- Coder
- Architecture
- Retrieval
- Augmented
- Generation
- AI
- LLM
- Mistral
- LLama
- Large Language Model
- Retrieval Augmented Generation
- Troy Andrew Schultz
- LookingForWork
- OpenForHire
- IdoCoolStuff
- Knowledge Graph
- Knowledge
- Graph
- Accelerator
- Enthusiast
- Chatbot
- Personal Assistant
- Copilot
- lol
- tags
- Pruned
- efficient
- smaller
- small
- local
- open
- source
- open source
- quant
- quantize
- ablated
- Ablation
- 'uncensored '
- unaligned
- 'bad '
- alignment
For those trying to shoe horn this large model on your machine every GB of saved memory counts when offloading to System RAM!
Here is a pruned down the 22.2 Billion parameter model by 2 junk layers to make a 21.5B that doesnt appear to lose any sense of quality.
- Created
- Oct 20, 2024
- Updated
- Oct 21, 2024