Downloads · 30 days
15
4% of all-time downloads
granite3-instruct-8b-gguf
trufae/granite3-instruct-8b-gguf
granite3-instruct-8b-gguf is a machine learning model from trufae. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
Source: ibm-granite/granite-3.0-8b-instruct
All-time downloads
424
Public
Repo size
20.9 GB
Likes
1
Public
Repo makeup
Click a slice to open those files.
.gguf20.9 GB · 100%
At a glance
- License
- apache-2.0
- Access
- Public
- Created
- Oct 22, 2024
- Updated
- Oct 22, 2024
- SHA
- 8c008093
Model card
From the Hugging Face model README
Granite3 GGUF
Source: ibm-granite/granite-3.0-8b-instruct
Setup:
git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
python -m venv venv
. venv/bin/activate
pip install .
Download the tensors:
from huggingface_hub import snapshot_download
model_id="ibm-granite/granite-3.0-8b-instruct"
snapshot_download(repo_id=model_id, local_dir="granite3",
local_dir_use_symlinks=False, revision="main")
Build the gguf:
python convert_hf_to_gguf.py --outfile granite3.gguf --outtype q8_0 granite3
Test with r2ai
r2ai -m "$PWD/granite3.gguf"
- License
- apache-2.0
- Created
- Oct 22, 2024
- Updated
- Oct 22, 2024