Downloads · 30 days
429
47% of all-time downloads
Vertex-0.6-100M-8192-Instruct-GGUF
VertexResearch/Vertex-0.6-100M-8192-Instruct-GGUF
Vertex-0.6-100M-8192-Instruct-GGUF is a text generation model from VertexResearch. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
GGUF quantizations of Vertex-0.6-100M-8192-Instruct for llama.cpp. 97M-parameter Qwen3-architecture chat model by Vertex Research: ChatML format, tool calling, 8192 context (RoPE theta 1M).
All-time downloads
911
Public
Repo size
931 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf931 MB · 100%
At a glance
- Task
- Text Generation
- Library
- llama.cpp
- License
- apache-2.0
- Access
- Public
- Created
- Aug 26, 2026
- Updated
- Nov 4, 2026
- SHA
- 3a2334f0
Try a prompt
Base models
Model card
From the Hugging Face model README
Vertex 0.6 100M — 8192-ctx Instruct (GGUF)
GGUF quantizations of Vertex-0.6-100M-8192-Instruct for llama.cpp. 97M-parameter Qwen3-architecture chat model by Vertex Research: ChatML format, tool calling, 8192 context (RoPE theta 1M).
Quants
| Quant | Size |
|---|---|
| BF16 | 195 MB |
| Q8_0 | 104 MB |
| Q5_K_M | 80 MB |
| Q5_K_S | 78 MB |
| Q4_K_M | 75 MB |
| Q4_K_S | 73 MB |
| Q4_1 | 70 MB |
| Q4_0 | 65 MB |
| Q3_K_M | 66 MB |
| Q3_K_S | 62 MB |
| Q2_K | 62 MB |
At 97M parameters the embedding table dominates file size, so low-bit quants save less than usual — and quantization hurts small models disproportionately. Q8_0 or higher recommended; BF16 for best quality.
Usage
llama-cli -m Vertex-0.6-100M-8192-Instruct-Q8_0.gguf
Chat template (ChatML) is embedded. EOS: </s> and <|im_end|>.
Limitations
These models are not the most coherent yet and need more tuning: expect rambling, repetition, and inconsistent answers, especially over longer generations.
- Task
- Text Generation
- Library
- llama.cpp
- License
- apache-2.0
- Languages
- en
- Created
- Aug 26, 2026
- Updated
- Nov 4, 2026