Hunyuan-1.8B-Instruct-Q4_K_M-GGUF

Edge-Quant/Hunyuan-1.8B-Instruct-Q4_K_M-GGUF

Hunyuan-1.8B-Instruct-Q4_K_M-GGUF is a machine learning model from Edge-Quant. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.

This model was converted to GGUF format from tencent/Hunyuan-1.8B-Instruct using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

Downloads · 30 days

8% of all-time downloads

All-time downloads

417

Public

Repo size

1.1 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.gguf1.1 GB · 100%

At a glance

Library: transformers
Access: Public
Created: Nov 28, 2025
Updated: Nov 30, 2025
SHA: e3b14825

Base models

tencent/Hunyuan-1.8B-Instruct

Model card

From the Hugging Face model README

Abhi99999/Hunyuan-1.8B-Instruct-Q4_K_M-GGUF

This model was converted to GGUF format from tencent/Hunyuan-1.8B-Instruct using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

Benchmark

Note: The following benchmarks are evaluated by TRT-LLM-backend on several base models.

Model	Hunyuan-0.5B-Pretrain	Hunyuan-1.8B-Pretrain	Hunyuan-4B-Pretrain	Hunyuan-7B-Pretrain
MMLU	54.02	64.62	74.01	79.82
MMLU-Redux	54.72	64.42	73.53	79
MMLU-Pro	31.15	38.65	51.91	57.79
SuperGPQA	17.23	24.98	27.28	30.47
BBH	45.92	74.32	75.17	82.95
GPQA	27.76	35.81	43.52	44.07
GSM8K	55.64	77.26	87.49	88.25
MATH	42.95	62.85	72.25	74.85
EvalPlus	39.71	60.67	67.76	66.96
MultiPL-E	21.83	45.92	59.87	60.41
MBPP	43.38	66.14	76.46	76.19
CRUX-O	30.75	36.88	56.5	60.75
Chinese SimpleQA	12.51	22.31	30.53	38.86
simpleQA (5shot)	2.38	3.61	4.21	5.69

Topic	Bench	Hunyuan-0.5B-Instruct	Hunyuan-1.8B-Instruct	Hunyuan-4B-Instruct	Hunyuan-7B-Instruct
Mathematics	AIME 2024<br>AIME 2025<br>MATH	17.2<br>20<br>48.5	56.7<br>53.9<br>86	78.3<br>66.5<br>92.6	81.1<br>75.3<br>93.7
Science	GPQA-Diamond<br>OlympiadBench	23.3<br>29.6	47.2<br>63.4	61.1<br>73.1	60.1<br>76.5
Coding	Livecodebench<br>Fullstackbench	11.1<br>20.9	31.5<br>42	49.4<br>54.6	57<br>56.3
Reasoning	BBH<br>DROP<br>ZebraLogic	40.3<br>52.8<br>34.5	64.6<br>76.7<br>74.6	83<br>78.2<br>83.5	87.8<br>85.9<br>85.1
Instruction<br>Following	IF-Eval<br>SysBench	49.7<br>28.1	67.6<br>55.5	76.6<br>68	79.3<br>72.7
Agent	BFCL v3<br> τ-Bench<br>ComplexFuncBench<br> C3-Bench	49.8<br>14.4<br>13.9<br>45.3	58.3<br>18.2<br>22.3<br>54.6	67.9<br>30.1<br>26.3<br>64.3	70.8<br>35.3<br>29.2<br>68.5
Long<br>Context	PenguinScrolls<br>longbench-v2<br>FRAMES	53.9<br>34.7<br>41.9	73.1<br>33.2<br>55.6	83.1<br>44.1<br>79.2	82<br>43<br>78.6

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Abhi99999/Hunyuan-1.8B-Instruct-Q4_K_M-GGUF --hf-file hunyuan-1.8b-instruct-q4_k_m.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Abhi99999/Hunyuan-1.8B-Instruct-Q4_K_M-GGUF --hf-file hunyuan-1.8b-instruct-q4_k_m.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Abhi99999/Hunyuan-1.8B-Instruct-Q4_K_M-GGUF --hf-file hunyuan-1.8b-instruct-q4_k_m.gguf -p "The meaning to life and the universe is"

./llama-server --hf-repo Abhi99999/Hunyuan-1.8B-Instruct-Q4_K_M-GGUF --hf-file hunyuan-1.8b-instruct-q4_k_m.gguf -c 2048

Library: transformers
Created: Nov 28, 2025
Updated: Nov 30, 2025