Llama-3.3-70B-Instruct-GGUF

BabaK07/Llama-3.3-70B-Instruct-GGUF

Llama-3.3-70B-Instruct-GGUF is a text generation model from BabaK07. Use it when you need the model to write or continue text. The card lists the license as llama3.3.

Using <a href="https://github.com/ggerganov/llama.cpp/"llama.cpp</a release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b4273"b4273</a for quantization.

Downloads · 30 days

7% of all-time downloads

All-time downloads

843

Public

Repo size

122 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf122 GB · 100%

At a glance

Task: Text Generation
License: llama3.3
Access: Public
Created: Feb 10, 2025
Updated: Feb 10, 2025
SHA: 3e902517

Try a prompt

Base models

meta-llama/Llama-3.3-70B-Instruct

Model card

From the Hugging Face model README

Llamacpp imatrix Quantizations of Llama-3.3-70B-Instruct

Using <a href="https://github.com/ggerganov/llama.cpp/">llama.cpp</a> release <a href="https://github.com/ggerganov/llama.cpp/releases/tag/b4273">b4273</a> for quantization.

Original model: https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct

Run them in LM Studio

Prompt format

<|begin_of_text|><|start_header_id|>system<|end_header_id|>

Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024

{system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>

{prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>

Download a file (not the whole branch) from below:

Filename	Quant type	File Size	Split	Description
Llama-3.3-70B-Instruct-Q5_K_M.gguf	Q5_K_M	49.9 GB	false	High quality, recommended.
Llama-3.3-70B-Instruct-Q4_K_M.gguf	Q4_K_M	42.5 GB	false	Good quality, default size for most use cases, recommended.
Llama-3.3-70B-Instruct-IQ3_XS.gguf	IQ3_XS	29.3 GB	false	Lower quality, new method with decent performance, slightly better than Q3_K_S.

Downloading using huggingface-cli

<details> <summary>Click to view download instructions</summary>

First, make sure you have hugginface-cli installed:

pip install -U "huggingface_hub[cli]"

Then, you can target the specific file you want:

huggingface-cli download BabaK07/Llama-3.3-70B-Instruct-GGUF --include "Llama-3.3-70B-Instruct-Q4_K_M.gguf" --local-dir ./

</details>

Task: Text Generation
License: llama3.3
Languages: en, fr, it, pt, hi, es
Created: Feb 10, 2025
Updated: Feb 10, 2025