Downloads · 30 days
559
62% of all-time downloads
Vertex-0.6-100M-8192-Instruct-v2-GGUF
VertexResearch/Vertex-0.6-100M-8192-Instruct-v2-GGUF
Vertex-0.6-100M-8192-Instruct-v2-GGUF is a text generation model from VertexResearch. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
GGUF conversion of VertexResearch/Vertex-0.6-100M-8192-Instruct-v2 for use with llama.cpp.
All-time downloads
897
Public
Repo size
1.1 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.gguf1.1 GB · 100%
At a glance
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Access
- Public
- Created
- Aug 29, 2026
- Updated
- Nov 4, 2026
- SHA
- 0f376c7c
Try a prompt
Base models
Model card
From the Hugging Face model README
Vertex 0.6 100M — 8192-ctx Instruct v2 (GGUF)
GGUF conversion of VertexResearch/Vertex-0.6-100M-8192-Instruct-v2 for use with llama.cpp.
Files
| File | Type |
|---|---|
Vertex-0.6-100M-8192-Instruct-v2-BF16.gguf | Full precision |
Vertex-0.6-100M-8192-Instruct-v2-F16.gguf | Full precision |
Vertex-0.6-100M-8192-Instruct-v2-Q8_0.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q5_K_M.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q5_K_S.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q4_K_M.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q4_K_S.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q4_1.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q4_0.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q3_K_M.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q3_K_S.gguf | Quantized |
Vertex-0.6-100M-8192-Instruct-v2-Q2_K.gguf | Quantized |
BF16/F16 are the recommended full-precision files (deliverable quality); the rest are size-reduced quants for constrained deployment — this model is tiny (97M params), so quantization mainly saves disk, not meaningfully more speed.
Model details
| Parameters | 96.75M (tied embeddings) |
| Architecture | Qwen3-based transformer |
| Context length | 8192 (RoPE theta 1M) |
| Chat format | ChatML (<|im_start|> / <|im_end|>) |
| Tool calling | <tool_call> JSON blocks, system-prompt function definitions |
| EOS | </s> (2) and <|im_end|> (6) |
Usage
llama-server -m Vertex-0.6-100M-8192-Instruct-v2-F16.gguf --host 0.0.0.0 --port 8080
or raw completion smoke test:
llama-completion -m Vertex-0.6-100M-8192-Instruct-v2-F16.gguf -p "The sky is" -n 16 --temp 0
Limitations
These models are not the most coherent yet and need more tuning: expect rambling, repetition, and inconsistent answers, especially over longer generations.
97M parameters: multi-turn chat is much improved and tool-call syntax works, but factual accuracy is low, reasoning is shallow, and it makes arithmetic errors. Not for production use. Knowledge cutoff ~April 2024.
Tool calling works mechanically (correct <tool_call> format, stops cleanly)
with a single available function, but multi-step tool use is unreliable: with
more than one function available it can pick the wrong tool, and it can
hallucinate details when summarizing a tool's response rather than reporting it
accurately. Don't trust it in an unsupervised agent loop.
- Task
- Text Generation
- Library
- gguf
- License
- apache-2.0
- Languages
- en
- Created
- Aug 29, 2026
- Updated
- Nov 4, 2026