Downloads · 30 days
65
3% of all-time downloads
Qwen3-0.6B-GGUF
mobilint/Qwen3-0.6B-GGUF
Qwen3-0.6B-GGUF is a text generation model from mobilint. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
<div align="center" <a href="https://mobilint.com" <img src="https://raw.githubusercontent.com/mobilint/.github/main/assets/MobilintLogoPrimary.png" width="50%" alt="mobilint" / </a </div
All-time downloads
2.1K
Public
Repo size
1.9 GB
Likes
2
Public
Repo makeup
Click a slice to open those files.
.bin622 MB · 50%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Model type
- mobilint-qwen3
- Access
- Public
- Created
- May 19, 2026
- Updated
- Oct 10, 2026
- SHA
- ecb0c7a2
Try a prompt
Base models
Model card
From the Hugging Face model README
<div align="center">
<a href="https://mobilint.com">
<img src="https://raw.githubusercontent.com/mobilint/.github/main/assets/Mobilint_Logo_Primary.png"
width="50%"
alt="mobilint" />
</a>
</div>
Qwen3-0.6B for llama.cpp-mblt
Pre-packaged bundle of Qwen3-0.6B for use with llama.cpp-mblt on Mobilint Regulus NPUs.
The transformer forward pass runs entirely on the NPU (.mxq body); the CPU handles tokenization, embedding lookup, chat templating, and sampling via llama.cpp.
Quickstart
# Auto-download and run
llama-cli-mblt -hf mobilint/Qwen3-0.6B-GGUF
# Or from a local directory
llama-cli-mblt -M /path/to/Qwen3-0.6B-GGUF
Works with every tool in the llama.cpp-mblt suite:
| Tool | Purpose |
|---|---|
llama-cli-mblt | Interactive chat / single-prompt CLI |
llama-simple-mblt | Minimal text generation |
llama-server-mblt | OpenAI-compatible HTTP server |
llama-bench-mblt | Prefill + decode throughput benchmark |
llama-perplexity-mblt | Perplexity on a text corpus |
llama-compare-mblt | Output comparison (KL divergence + cosine) |
Contents
| File | Purpose |
|---|---|
Qwen3-0.6B-W8.mxq | Pre-compiled NPU model (W8 quantized, Single-IO, Regulus single-core) |
qwen3-0.6b-vocab.gguf | Vocab-only GGUF (tokenizer + chat template) |
target_emb.bin | Input embedding matrix (float32, shape [vocab=151936, hidden=1024]) |
config.json | Model geometry (hidden_size, layers, RoPE θ, etc.) |
proxy_qwen3.py | Optional HF transformers proxy class |
README.md | This file |
Format
- Model format: Single-IO (input: embeddings; output: logits). RoPE and attention masks are baked into the MXQ.
- Core mode:
single(Regulus has 1 cluster × 1 local core). - Quantization: W8 (8-bit weights).
- EAGLE3: Not supported for Qwen3 (Single-IO format).
- Context length: 16 384 tokens (per
config.json).
Hardware & Software Requirements
- Mobilint Regulus NPU (e.g., regulus2 board)
- llama.cpp-mblt built against qbruntime ≥ v1.2.0 / regulus-release v3.4.0
- NPU driver installed on host (see docs.mobilint.com)
This bundle was prepared with regulus-release v3.4.0 (qbruntime v1.2.0).
Source Model
This is a quantized + NPU-compiled derivative of Qwen/Qwen3-0.6B. The original model is © Alibaba Cloud and is distributed under the Apache 2.0 license — see the upstream LICENSE for terms.
The unquantized Mobilint source model with .mxq + .safetensors is available at mobilint/Qwen3-0.6B; this repo is the llama.cpp-mblt-ready bundle.
- Task
- Text Generation
- Library
- gguf
- Type
- mobilint-qwen3
- License
- apache-2.0
- Created
- May 19, 2026
- Updated
- Oct 10, 2026