Qwen_Qwen3-4B-GGUF

Qwen 3 4B

[!NOTE] Qwen 3 is not the latest model in the Qwen series. Qwen 3.5 is now available.

---

Model Capabilities

• Text generation — instruction-following chat model
• Tool calling — supports function/tool call syntax
• Thinking mode — can show explicit reasoning steps before answering (see Getting Started)
• Multilinguality — supports over 100 different languages

The full description can be found at the original model page.

---

Getting Started

Install NobodyWho:

pip install nobodywho

Run with NobodyWho — the model is downloaded and cached automatically on first use:

from nobodywho import Chat

chat = Chat("huggingface:NobodyWho/Qwen_Qwen3-4B-GGUF/Qwen_Qwen3-4B-Q4_K_M.gguf")
response = chat.ask("What is the capital of Denmark?").completed()
print(response) # Copenhagen!

To enable thinking mode:

chat = Chat(
    "huggingface:NobodyWho/Qwen_Qwen3-4B-GGUF/Qwen_Qwen3-4B-Q4_K_M.gguf",
    template_variables={"enable_thinking": True},
)
response = chat.ask("Solve this step by step: if x + 3 = 7, what is x?").completed()
print(response) # 4

Prefer a manual download (e.g. for the Godot or Flutter binding)? Grab the file through the huggingface UI, or:

wget https://huggingface.co/NobodyWho/Qwen_Qwen3-4B-GGUF/resolve/main/Qwen_Qwen3-4B-Q4_K_M.gguf

and pass the local path instead: Chat("./Qwen_Qwen3-4B-Q4_K_M.gguf").

---

Benchmarks

Coming soon.

---

How are these GGUFs different?

These GGUF files serve mainly for NobodyWho inference library and are a way that we can guarantee that they will include all of the necessary info to be truly portable (sampler config, token descriptions, etc.). It is our informal effort to somehow arrive at more standardized GGUF files, which contain everything the runtime needs so that the user can have effortless experience using them, without additional fiddling or patching stuff up for every single model.

---

Model Details

---

Credits

GGUF quantizations provided by bartowski. Thanks!