cot_5k-GGUF

snakech/cot_5k-GGUF

cot_5k-GGUF is a text generation model from snakech. Use it when you need the model to write or continue text. It is set up for transformers.

This repo contains GGUF format model files for FabienRoger/cot5k.

Downloads · 30 days

8% of all-time downloads

All-time downloads

387

Public

Repo size

694 MB

Likes

Public

Repo makeup

Click a slice to open those files.

.gguf694 MB · 100%

From the Hugging Face model README

This repo contains GGUF format model files for FabienRoger/cot_5k.

they are compatible with llama.cpp as of commit b4011.

<|system|>
{system_prompt}<|endoftext|>
<|user|>
{prompt}<|endoftext|>
<|assistant|>

Filename	Quant type	File Size	Description
cot_5k-Q2_K.gguf	Q2_K	0.646 GB	smallest, significant quality loss - not recommended for most purposes

Firstly, install Huggingface Client

pip install -U "huggingface_hub[cli]"

Then, downoad the individual model file the a local directory

huggingface-cli download tensorblock/cot_5k-GGUF --include "cot_5k-Q2_K.gguf" --local-dir MY_LOCAL_DIR

If you wanna download multiple model files with a pattern (e.g., *Q4_K*gguf), you can try:

huggingface-cli download tensorblock/cot_5k-GGUF --local-dir MY_LOCAL_DIR --local-dir-use-symlinks False --include='*Q4_K*gguf'