Downloads · 30 days
43
7% of all-time downloads
granite-claude-h-350m-GGUF
Avtrkrb/granite-claude-h-350m-GGUF
granite-claude-h-350m-GGUF is a text generation model from Avtrkrb. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
All-time downloads
623
Public
Repo size
2.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf2.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Access
- Public
- Created
- Jun 11, 2026
- Updated
- Jun 11, 2026
- SHA
- 7c23ebe0
Try a prompt
Base models
Model card
From the Hugging Face model README
granite-claude-h-350m-GGUF
GGUF quantizations of:
Avtrkrb/granite-claude-h-350m
These files are intended for inference using:
- llama.cpp
- LM Studio
- Open WebUI
- Jan
- KoboldCpp
- GPT4All
- Ollama (after conversion/import)
Available Quantizations
Typical variants included:
| Quant | Use Case |
|---|---|
| Q4_K_M | Best size / quality balance |
| Q5_K_M | Higher quality |
| Q6_K | Near-lossless for most use cases |
| Q8_0 | Highest quality quantized version |
Source Model
Merged model:
https://huggingface.co/Avtrkrb/granite-claude-h-350m
Dataset:
https://huggingface.co/datasets/Avtrkrb/combined-reasoning-claude
Example llama.cpp Usage
./llama-cli \
-m granite-claude-h-350m-Q4_K_M.gguf \
-p "Explain quantum tunneling."
Recommended Quant
For most users:
Q4_K_M
offers the best balance between:
- quality
- speed
- memory usage
License
This repository follows the licensing terms of the original Granite model.
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Languages
- en
- Created
- Jun 11, 2026
- Updated
- Jun 11, 2026