Downloads · 30 days
0
0% of all-time downloads
TheHouseOfTheDude/L3.3-70B-Animus-V12.0_Compressed-Tensors
L3.3-70B-Animus-V12.0_Compressed-Tensors is a text generation model from TheHouseOfTheDude. Use it when you need the model to write or continue text. It is set up for vllm. The card lists the license as other.
This repository hosts quantized runtime builds of Darkhn/Magistral-2509-24B-Animus-V12.0, repackaged for vLLM using the compressed-tensors format.
Downloads · 30 days
0
0% of all-time downloads
All-time downloads
41
Public
Repo size
113 GB
Likes
0
Public
Click a slice to open those files.
.md2.9 KB · 66%
From the Hugging Face model README
This repository hosts quantized runtime builds of
Darkhn/Magistral-2509-24B-Animus-V12.0, repackaged for vLLM using the compressed-tensors format.
TL;DR
• Quantized package intended for vLLM with--quantization compressed-tensors.
• Two published branches: W4A16 (INT4/A16) and W8A16 (INT8/A16).
• Weight-only quantization; see “Quantization recipe” below.
The
mainbranch is a landing page (model card + links). All runnable artifacts live under per-revision branches.
Quick links
*.safetensors) + index (model.safetensors.index.json)config.json with compressed-tensors metadata (weight_format, quantization, quantization_config, etc.)tokenizer.json, tokenizer.model, merges/vocab if applicable)chat_template.jinja (inherits the parent finetune’s chat style)Exact files vary by branch; see the Files and versions tab.
Shared choices
lm_head kept in higher precisionsave_compressed=True to embed compressed-tensors metadata for vLLMBranch specifics
Refer to each branch’s
config.jsonfor the exact parameters used.
Install vLLM (recent version recommended):
pip install vllm
Serve (adjust to your hardware):
CUDA_VISIBLE_DEVICES=0,1,2,3 \
vllm serve TheHouseOfTheDude/L3.3-70B-Animus-V12.0_Compressed-Tensors \
--quantizat