Grok-1-GGUF

Arki05/Grok-1-GGUF

Grok-1-GGUF is a machine learning model from Arki05. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.

This repository contains unofficial GGUF Quantizations of Grok-1, compatible with llama.cpp as of PR- Add grok-1 support 6204.

Downloads · 30 days

22.9K

4% of all-time downloads

All-time downloads

589K

Public

Repo size

3.9 TB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf3.8 TB · 100%

At a glance

Library: transformers
License: apache-2.0
Model type: grok
Access: Public
Created: Mar 21, 2024
Updated: Apr 11, 2024
SHA: d4359f59

Model card

From the Hugging Face model README

Grok-1 GGUF Quantizations

This repository contains unofficial GGUF Quantizations of Grok-1, compatible with llama.cpp as of PR- Add grok-1 support #6204.

Updates

Native Split Support in llama.cpp

The splits have been updated to utilize the improvements from PR: llama_model_loader: support multiple split/shard GGUFs. As a result, manual merging with gguf-split is no longer required.

With this, there is no need to merge the split files before use. Just download all splits and run llama.cpp with the first split like you would previously. It'll detect the other splits and load them as well.

Direct Split Download from huggingface using llama.cpp

Thanks to a new PR common: llama_load_model_from_url split support #6192 from phymbert it's now possible load model splits from url.

That means this downloads and runs the model:

server \
    --hf-repo Arki05/Grok-1-GGUF \
    --hf-file grok-1-IQ3_XS-split-00001-of-00009.gguf \
    --model models/grok-1-IQ3_XS-split-00001-of-00009.gguf \
    -ngl 999

And that is very cool (@phymbert)

Available Quantizations

The following Quantizations are currently available for download:

Quant	Split Files	Size
`Q2_K`	1-of-9, 2-of-9, 3-of-9, 4-of-9, 5-of-9, 6-of-9, 7-of-9, 8-of-9, 9-of-9	112.4 GB
`IQ3_XS`	1-of-9, 2-of-9, 3-of-9, 4-of-9, 5-of-9, 6-of-9, 7-of-9, 8-of-9, 9-of-9	125.4 GB
`Q4_K`	1-of-9, 2-of-9, 3-of-9, 4-of-9, 5-of-9, 6-of-9, 7-of-9, 8-of-9, 9-of-9	186.0 GB
`Q6_K`	1-of-9, 2-of-9, 3-of-9, 4-of-9, 5-of-9, 6-of-9, 7-of-9, 8-of-9, 9-of-9	259.8 GB

I would recommend the IQ3_XS version for now.

More Quantizations will be uploaded soon. All current Quants are created without any importance matrix.

Library: transformers
Type: grok
License: apache-2.0
Created: Mar 21, 2024
Updated: Apr 11, 2024