Downloads · 30 days
11
22% of all-time downloads
Ful3/Dark-Scarlett-v1.0-31B-hybrid12-multilingual
Dark-Scarlett-v1.0-31B-hybrid12-multilingual is a machine learning model from Ful3. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for vllm.
Mixed-precision quantization of ReadyArt/Dark-Scarlett-v1.0-31B (Gemma 4 31B) made with llm-compressor, for vLLM.
Downloads · 30 days
11
22% of all-time downloads
All-time downloads
49
Public
Parameters
20.2B
27.8 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors27.7 GB · 100%
How the weights are stored.
F8_E4M316.6B · 82%
From the Hugging Face model README
Mixed-precision quantization of ReadyArt/Dark-Scarlett-v1.0-31B (Gemma 4 31B) made with llm-compressor, for vLLM.
q/k/v/o_proj): 4-bit GPTQ, weight-only (W4A16), group size 32gate/up/down_proj): FP8 dynamic (W8A8) in 48 layers; 4-bit GPTQ (W4A16, group 32) in layers 10, 14, 18, 22, 26, 30, 34, 38, 42, 46, 50, 54Multilingual calibration. GPTQ was calibrated on 320 conversations at 4096 tokens: 96 English roleplay conversations plus 224 self-play roleplay conversations written by the BF16 model itself in Spanish, German, Italian, French, Korean, Japanese (28 each), Czech (20), Polish, Portuguese and Russian (12 each). An English-only calibration of the same recipe leaked English words into other languages and dropped Spanish accents; this build does not. On a 92-reply companion-chat check in 8 languages it matches the BF16 model (no English leaks, no wrong-script characters, no degenerate replies).
vllm serve <this-repo> --language-model-only --kv-cache-dtype fp8
License and usage terms follow the base model (Gemma terms).