Downloads · 30 days
131
5% of all-time downloads
Yi-34B-Llama-GGUF
simustar/Yi-34B-Llama-GGUF
Yi-34B-Llama-GGUF is a text generation model from simustar. Use it when you need the model to write or continue text.
This model repository contains files in GGUF format for the Yi 34B LLaMA, compatible with LLaMA modeling, based on the work from the chargoddard/Yi-34B-Llama repository.
All-time downloads
2.7K
Public
Repo size
217 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf217 GB · 100%
At a glance
- Task
- Text Generation
- Access
- Public
- Created
- Nov 8, 2023
- Updated
- Nov 9, 2023
- SHA
- b768b4ed
Try a prompt
Model card
From the Hugging Face model README
This model repository contains files in GGUF format for the Yi 34B LLaMA, compatible with LLaMA modeling, based on the work from the chargoddard/Yi-34B-Llama repository.
Based of the work of chargoddard's:
- Tensors have been renamed to match the standard LLaMA.
- Model can be loaded without trust_remote_code, but the tokenizer can not.
Converted & Quantized Files
Yi-34B-Llamafied Model Options
The following tables list the available Yi-34B-Llamafied model files with their respective quantization methods and characteristics.
Key:
- Size: File size relative to the original.
- Quality Loss: The amount of quality loss due to quantization.
| Q-Method | File Name | Size | Quality Loss | Recommended |
|---|---|---|---|---|
| Q2 | Yi-34B-Llama_Q2_K | Smallest | Extreme (not recommended) | |
| Q3 | Yi-34B-Llama_Q3_K_S | Very Small | Very High | |
| Q3 | Yi-34B-Llama_Q3_K_M | Very Small | Very High | |
| Q3 | Yi-34B-Llama_Q3_K_L | Small | Substantial | |
| Q4 | Yi-34B-Llama_Q4_K_S | Small | Significant | |
| Q4 | Yi-34B-Llama_Q4_K_M | Medium | Balanced | Recommended |
| Q5 | Yi-34B-Llama_Q5_K_S | Large | Low | Recommended |
| Q5 | Yi-34B-Llama_Q5_K_M | Large | Very Low | Recommended |
| Q6 | Yi-34B-Llama_Q6_K | Very Large | Extremely Low | |
| Q8 | Yi-34B-Llama_Q8_0 | Very Large | Extremely Low (not recommended) |
Please choose the model that best suits your needs based on the size and quality loss trade-offs.
- Task
- Text Generation
- Languages
- en, zh
- Created
- Nov 8, 2023
- Updated
- Nov 9, 2023