Downloads · 30 days
31
12% of all-time downloads
purrgpt-community/Tiny-Purr-350M-GGUF
Tiny-Purr-350M-GGUF is a text generation model from purrgpt-community. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
This model is a GGUF conversion of purrgpt-community/Tiny-Purr-350M-merged, created using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details.
Downloads · 30 days
31
12% of all-time downloads
All-time downloads
251
Public
Repo size
459 MB
Likes
0
Public
Click a slice to open those files.
.gguf229 MB · 100%
From the Hugging Face model README
This model is a GGUF conversion of purrgpt-community/Tiny-Purr-350M-merged, created using llama.cpp via the ggml.ai's GGUF-my-repo space.
Refer to the original model card for more details.
Install llama.cpp (works on Mac and Linux):
brew install llama.cpp
You can invoke the model using either the CLI or the server.
llama-cli --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -p "The meaning to life and the universe is"
llama-server --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -c 2048
git clone https://github.com/ggerganov/llama.cpp
LLAMA_CURL=1 flag. Add hardware-specific flags if needed (e.g., LLAMA_CUDA=1 for Nvidia GPUs on Linux):cd llama.cpp
LLAMA_CURL=1 make
./llama-cli --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -p "The meaning to life and the universe is"
or
./llama-server --hf-repo purrgpt-community/Tiny-Purr-350M-GGUF --hf-file tiny-purr-350m-q4_k_m.gguf -c 2048