Downloads · 30 days
70
9% of all-time downloads
0x795E2795/Vanta-Alpha-26B-A4B-it-qat-q4_0
Vanta-Alpha-26B-A4B-it-qat-q4_0 is a text generation model from 0x795E2795. Use it when you need the model to write or continue text.
Vanta Alpha is an experimental 26B-class sparse Mixture-of-Experts language model with approximately 4B active parameters per token.
Downloads · 30 days
70
9% of all-time downloads
All-time downloads
822
Public
Repo size
15.4 GB
Likes
0
Public
Click a slice to open those files.
.gguf15.4 GB · 100%
From the Hugging Face model README
Vanta Alpha is an experimental 26B-class sparse Mixture-of-Experts language model with approximately 4B active parameters per token.
This repository provides the model in GGUF Q4_0 format for lightweight local inference and compatibility testing with GGUF-based runtimes.
This release is intended for early testing, independent evaluation, runtime compatibility testing, and community feedback.
Alpha release: model behavior, compatibility, formatting, and weights may change in future revisions.
| Property | Value |
|---|---|
| Model | Vanta Alpha 26B-A4B |
| Architecture | Sparse Mixture-of-Experts |
| Total parameters | 26B-class |
| Active parameters | ~4B |
| Layers | 30 |
| Experts | 128 |
| Routed experts | Top-8 |
| Context length | 262,144 tokens |
| Sliding attention window | 1,024 tokens |
| Format | GGUF |
| Quantization | Q4_0 |
| Release status | Alpha / Experimental |
Parameter counts reported by individual libraries or hosting platforms may differ depending on whether embeddings, auxiliary parameters, and other components are included.
This repository uses the Q4_0 GGUF quantization format.
Q4_0 is intended to provide a compact model representation with broad support across GGUF-compatible inference engines.
This checkpoint is primarily provided for:
Because quantization changes numerical precision, outputs may differ from other Vanta Alpha checkpoints even when identical prompts and generation settings are used.
Vanta Alpha is configured for a maximum context length of:
262,144 tokens
The architecture also uses a local sliding attention window of:
1,024 tokens
Actual usable context length depends heavily on:
Very-long-context behavior remains experimental.
A reasonable starting point is:
temperature: 1.225
top_p: 0.95
top_k: 64
These values are not strict requirements.
For benchmark evaluation, use the generation settings required by the benchmark rather than these defaults.
Example:
./llama-cli \
-m vanta-alpha-26B_q4_0-it.gguf \
-cnv \
-c 8192 \
--temp 1.225 \
--top-p 0.95 \
--top-k 64
For GPU offloading:
./llama-cli \
-m vanta-alpha-26B_q4_0-it.gguf \
-cnv \
-c 8192 \
-ngl 999 \
--temp 1.225 \
--top-p 0.95 \
--top-k 64
Adjust context length and GPU offloading according to your hardware.
Support may depend on the version of llama.cpp or other GGUF-compatible runtimes being used.
Vanta Alpha can also be served through a compatible llama.cpp server build.
./llama-server \
-m vanta-alpha-26B_q4_0-it.gguf \
-c 8192 \
-ngl 999 \
--host 0.0.0.0 \
--port 8080
Runtime flags may differ between versions.
Memory usage depends on more than the GGUF file size.
Additional memory is required for:
Long context lengths can significantly increase memory consumption even when the model weights themselves fit comfortably in memory.
CPU-only inference is possible in compatible runtimes, but performance will depend strongly on system memory bandwidth and CPU capabilities.
Benchmark evaluation is currently in progress.
Results will be published after the evaluation suite has been completed and the measurements have been verified.
The evaluation is intended to cover areas including:
Preliminary or incomplete benchmark numbers are intentionally not included in this README.
Different quantization formats should not be assumed to behave identically.
When comparing this Q4_0 release against other checkpoints, useful measurements include:
A single accuracy score may not fully describe quantization quality.
This release is primarily intended to collect real-world observations across different hardware and inference environments.
Useful feedback includes:
When reporting performance, using tok/s for generation speed is preferred.
Vanta Alpha is an experimental checkpoint.
Current limitations may include:
Outputs should not be assumed to be bit-for-bit or behaviorally identical across different inference engines.
Vanta Alpha GGUF is intended for:
As an Alpha release, it should be evaluated carefully before use in reliability-sensitive applications.
This model follows the Gemma 4 license and usage terms.
Users are responsible for reviewing and complying with the applicable Gemma license terms and usage restrictions when downloading, modifying, redistributing, or using this model.
Vanta Alpha is provided as an experimental model release.
Generated outputs may be inaccurate, inconsistent, biased, or otherwise unsuitable for a particular use case.
Users should independently verify important outputs and evaluate the model for their intended application.
Bug reports, compatibility results, benchmark measurements, and other technical feedback are welcome.
When reporting results, please include enough runtime and hardware information to reproduce the test where possible.