Downloads · 30 days
15
2% of all-time downloads
rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm
Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm is a image-text-to-text model from rdtand. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for transformers. The card lists the license as apache-2.0.
This is a PrismaQuant mixed-precision compressed-tensors export of llmfan46/Qwen3.6-27B-uncensored-heretic-v2 for vLLM serving.
Downloads · 30 days
15
2% of all-time downloads
All-time downloads
954
Public
Parameters
18.9B
22 GB on disk
Likes
1
Public
Click a slice to open those files.
.safetensors22 GB · 100%
How the weights are stored.
U89.7B · 51%
From the Hugging Face model README
This is a PrismaQuant mixed-precision compressed-tensors export of
llmfan46/Qwen3.6-27B-uncensored-heretic-v2
for vLLM serving.
Validated locally with vLLM load and greedy generation:
The capital of France isParis.The serving run selected:
CutlassFP8ScaledMMLinearKernel for compressed-tensors FP8FlashInferCutlassNvFp4LinearKernel for NVFP4 GEMMNo KL quality claim is made in this model card.
vllm serve rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm \
--quantization compressed-tensors \
--trust-remote-code
For a low-context smoke:
python -m vllm.entrypoints.openai.api_server \
--model rdtand/Qwen3.6-27B-PrismaQuant-Heretic-5.25bit-vllm \
--quantization compressed-tensors \
--trust-remote-code \
--max-model-len 528
llmfan46/Qwen3.6-27B-uncensored-heretic-v2bf309da6e5744ac57240ae21d31eaf107734c585/home/rob/dq-runs/qwen36-27b-heretic-v2-kneedle-5p25-materialize-20260515T220544Z/exported