Downloads · 30 days
20
5% of all-time downloads
JongYeop/Qwen3-30B-A3B-FP8-W8A8
Qwen3-30B-A3B-FP8-W8A8 is a text generation model from JongYeop. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Qwen/Qwen3-30B-A3B quantized to FP8 W8A8 (static per-tensor weights + activations) with llm-compressor.
Downloads · 30 days
20
5% of all-time downloads
All-time downloads
418
Public
Parameters
30.5B
31.2 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors31.2 GB · 100%
How the weights are stored.
F8_E4M329.9B · 98%
From the Hugging Face model README
Qwen/Qwen3-30B-A3B quantized to FP8 W8A8 (static per-tensor weights + activations) with llm-compressor.
This is a Mixture-of-Experts (MoE) model. The MoE router (mlp.gate) and the
lm_head are kept at full precision; the attention and expert projection layers
are quantized to FP8.
lm_head, MoE router (re:.*mlp.gate$)HuggingFaceH4/ultrachat_200k
(train_sft), max sequence length 2048from vllm import LLM, SamplingParams
llm = LLM(model="JongYeop/Qwen3-30B-A3B-FP8-W8A8")
out = llm.generate(
["Explain mixture-of-experts in one sentence."],
SamplingParams(temperature=0.7, max_tokens=128),
)
print(out[0].outputs[0].text)
quant_stage:
quant_modifiers:
QuantizationModifier:
ignore: ["lm_head", "re:.*mlp.gate$"]
config_groups:
group_0:
weights:
num_bits: 8
type: float
strategy: tensor
dynamic: false
symmetric: true
input_activations:
num_bits: 8
type: float
strategy: tensor
dynamic: false
symmetric: true
targets: ["Linear"]