Downloads · 30 days
22
8% of all-time downloads
0x795E2795/Vanta-Alpha-26B-A4B-it-qat-w4a16-ct
Vanta-Alpha-26B-A4B-it-qat-w4a16-ct is a text generation model from 0x795E2795. Use it when you need the model to write or continue text.
Vanta Alpha is an experimental 26B-class sparse Mixture-of-Experts language model with approximately 4B active parameters per token, released as a W4A16 QAT checkpoint.
Downloads · 30 days
22
8% of all-time downloads
All-time downloads
292
Public
Parameters
27.9B
19.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors19.2 GB · 100%
How the weights are stored.
I3226.6B · 95%
From the Hugging Face model README
Vanta Alpha is an experimental 26B-class sparse Mixture-of-Experts language model with approximately 4B active parameters per token, released as a W4A16 QAT checkpoint.
This release is intended for early testing, inference compatibility testing, independent evaluation, and community feedback.
Alpha release: model behavior, compatibility, formatting, and weights may change in future revisions.
| Property | Value |
|---|---|
| Model | Vanta Alpha 26B-A4B |
| Architecture | Sparse Mixture-of-Experts |
| Total parameters | 26B-class |
| Active parameters | 3.6B |
| Layers | 30 |
| Experts | 128 |
| Routed experts | Top-8 |
| Context length | 262,144 tokens |
| Sliding attention window | 1,024 tokens |
| Weight format | W4A16 |
| Quantization | Quantization-Aware Training |
| Weight precision | INT4 |
| Activation precision | BF16 |
| Quantization group size | 32 |
| Quantization format | compressed-tensors |
| Release status | Alpha / Experimental |
Parameter counts reported by individual libraries or model hosting platforms may differ slightly depending on whether embeddings, auxiliary parameters, and other components are included in the total.
Vanta Alpha is released as a Quantization-Aware Trained (QAT) model rather than a conventional post-training quantized checkpoint.
The weights use symmetric 4-bit group quantization with a group size of 32 through the compressed-tensors format, while activations remain at higher precision.
The goal of this release is not only to reduce model size and inference memory requirements, but also to preserve the behavior of the higher-precision model as closely as practical under low-bit inference.
Vanta Alpha supports a maximum configured context length of:
262,144 tokens
The architecture also uses a local sliding attention window of:
1,024 tokens
Very-long-context behavior is still considered experimental and may vary depending on inference backend, available memory, KV-cache implementation, and workload.
The default generation configuration provides a reasonable starting point:
temperature = 1.225
top_p = 0.95
top_k = 64
These values are not strict requirements.
For deterministic or benchmark-oriented evaluation, use the sampling configuration required by the benchmark rather than these defaults.
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "0x795E2795/Vanta-Alpha-26B-A4B-it-qat-w4a16-ct"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
torch_dtype="auto",
)
messages = [
{
"role": "user",
"content": "Explain quantization-aware training in simple terms."
}
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(
inputs,
max_new_tokens=512,
temperature=1.0,
top_p=0.95,
top_k=64,
do_sample=True,
)
response = tokenizer.decode(
outputs[0][inputs.shape[-1]:],
skip_special_tokens=True,
)
print(response)
Inference support for low-bit compressed-tensors models may depend on the backend and software version being used.
Benchmark evaluation is currently in progress.
Results will be published after the evaluation suite has been completed and the measurements have been verified.
The evaluation is intended to cover areas including:
Preliminary or incomplete benchmark numbers are intentionally not included in this model card.
This release is primarily intended to collect real-world observations from different inference environments and workloads.
Useful feedback includes:
Independent benchmark results are also welcome.
Vanta Alpha is an experimental checkpoint.
Current limitations may include:
Outputs should not be assumed to be bit-for-bit or behaviorally identical across different runtimes.
Vanta Alpha is intended for:
As an Alpha release, it should be evaluated carefully before use in reliability-sensitive applications.
This model follows the Gemma 4 license and usage terms.
Users are responsible for reviewing and complying with the applicable Gemma license terms and usage restrictions when downloading, modifying, redistributing, or using this model.
Vanta Alpha is provided as an experimental model release.
Generated outputs may be inaccurate, inconsistent, biased, or otherwise unsuitable for a particular use case. Users should independently verify important outputs and evaluate the model for their intended application.
Bug reports, compatibility results, benchmark measurements, and other technical feedback are welcome.
When reporting results, please include enough information to reproduce the test where possible.