Downloads · 30 days
211
38% of all-time downloads
Vertex-0.6-35M-Instruct-GGUF
VertexResearch/Vertex-0.6-35M-Instruct-GGUF
Vertex-0.6-35M-Instruct-GGUF is a text generation model from VertexResearch. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
GGUF quantizations of VertexResearch/Vertex-0.6-35M-Instruct, a ≈34M-parameter Qwen3-architecture chat model. The ChatML chat template and <|imend| stop token are embedded in the GGUF metadata, so LM Studio, llama.cpp…
All-time downloads
555
Public
Repo size
356 MB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf356 MB · 100%
At a glance
- Task
- Text Generation
- Library
- llama.cpp
- License
- apache-2.0
- Access
- Public
- Created
- Aug 25, 2026
- Updated
- Aug 26, 2026
- SHA
- 9d84a068
Try a prompt
Base models
Model card
From the Hugging Face model README
Vertex-0.6-35M-Instruct — GGUF
GGUF quantizations of VertexResearch/Vertex-0.6-35M-Instruct,
a ≈34M-parameter Qwen3-architecture chat model. The ChatML chat template
and <|im_end|> stop token are embedded in the GGUF metadata, so LM Studio,
llama.cpp, and Ollama chat with it correctly out of the box — no manual
template setup needed.
Files
| Quant | Size | Notes |
|---|---|---|
| BF16 | 69 MB | Full precision |
| Q8_0 | 37 MB | Recommended — at this model size there is little reason to go lower |
| Q5_K_M | 31 MB | |
| Q5_K_S | 30 MB | |
| Q4_K_M | 29 MB | |
| Q4_K_S | 28 MB | |
| Q4_1 | 28 MB | Legacy |
| Q4_0 | 26 MB | Legacy |
| Q3_K_M | 27 MB | Quality loss noticeable on a model this small |
| Q3_K_S | 26 MB | Quality loss noticeable on a model this small |
| Q2_K | 26 MB | Not recommended at 34M params |
Note: the model's hidden size (384) is not a multiple of 256, so k-quants fall back to legacy formats for some tensors — the sub-Q4 files save less space than usual and Q8_0 remains the best pick.
Quick start
- LM Studio: search for this repo, pick a quant (Q8_0 recommended), download, and chat — the template is auto-detected.
- llama.cpp:
llama-cli -m Vertex-0.6-35M-Instruct-Q8_0.gguf - Ollama:
ollama run hf.co/VertexResearch/Vertex-0.6-35M-Instruct-GGUF:Q8_0
Limitations
These models are not the most coherent yet and need more tuning: expect rambling, repetition, and inconsistent answers, especially over longer generations.
34M parameters: simple conversational ability only — expect weak factual reliability, arithmetic, and instruction-following on complex rewrites. English-centric, 1024-token context, no safety tuning.
- Task
- Text Generation
- Library
- llama.cpp
- License
- apache-2.0
- Languages
- en
- Created
- Aug 25, 2026
- Updated
- Aug 26, 2026