Downloads · 30 days
11
20% of all-time downloads
Legal-Qwen3.6-27B-Abliterated-FP8
scottyjmp5/Legal-Qwen3.6-27B-Abliterated-FP8
Legal-Qwen3.6-27B-Abliterated-FP8 is a image-text-to-text model from scottyjmp5. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
FP8 W8A8 (dynamic per-token activation) quantization of Legal-Qwen3.6-27B-Abliterated, produced with llm-compressor.
All-time downloads
55
Public
Parameters
27.4B
30.4 GB on disk
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors30.4 GB · 100%
Parameter types
How the weights are stored.
F8_E4M324.4B · 89%
Try a prompt
Base models
Model card
From the Hugging Face model README
Legal-Qwen3.6-27B-Abliterated-FP8
FP8 W8A8 (dynamic per-token activation) quantization of Legal-Qwen3.6-27B-Abliterated, produced with llm-compressor.
- Text Linear layers are FP8; the vision tower, embeddings, and lm_head are kept full precision, so vision, tool calling, and thinking mode are unaffected.
- 29 GB (vs 52 GB bf16). Serves on a single 40GB+ GPU with vLLM (compressed-tensors is detected automatically). Fits 2x24GB with tensor parallelism.
- FP8 W8A8 uses precompiled CUTLASS kernels in vLLM: no custom toolchain needed.
vllm serve scottyjmp5/Legal-Qwen3.6-27B-Abliterated-FP8 \
--trust-remote-code --max-model-len 8192
Tip observed in testing: FP8's speed advantage over bf16 appears when CUDA graphs are enabled (the vLLM default). With enforce_eager it can be slightly slower than bf16; with graphs it was 1.67x faster in our single-stream tests.
See the main model card for training details, the retrieval (RAG) recommendation, and warnings (abliterated base; not legal advice).
- Task
- Image-Text-to-Text
- Type
- qwen3_5
- License
- apache-2.0
- Languages
- en
- Created
- Jul 16, 2026
- Updated
- Jul 16, 2026