Downloads · 30 days
45
61% of all-time downloads
pt810/Ovis-Omni-Embedding-3B-gptq-mixed-w2-w4-w8
Ovis-Omni-Embedding-3B-gptq-mixed-w2-w4-w8 is a text-to-audio model from pt810. Use it for the text-to-audio task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Experimental calibration-based GPTQ-derived variant of ATH-MaaS/Ovis-Omni-Embedding-3B.
Downloads · 30 days
45
61% of all-time downloads
All-time downloads
74
Public
Parameters
2.9K
7.7 GB on disk
Likes
0
Public
Click a slice to open those files.
.safetensors7.6 GB · 99%
How the weights are stored.
BF163.4B · 68%
From the Hugging Face model README
Experimental calibration-based GPTQ-derived variant of ATH-MaaS/Ovis-Omni-Embedding-3B.
Thinker layers 0–13 use 2-bit weights, layers 14–26 use 4-bit weights, and layer 27 uses 8-bit weights. The multimodal towers and non-linear/special modules remain in the original precision. Group scales were recomputed from the original BF16 weights after the compressor export produced invalid scales.
Compatibility warning: this W2 mixed artifact is published for experimentation, but vLLM 0.30.0 rejects its W2 packed shape in the Marlin compressed-tensors loader. Do not treat it as a vLLM-servable checkpoint. Use the separately published W4/W4/W8 vLLM-tested artifact for vLLM serving.