Downloads · 30 days
31
15% of all-time downloads
Wondernutts/Qwen3.8-27B-Uncensored-int4-awq-g128-ov
Qwen3.8-27B-Uncensored-int4-awq-g128-ov is a image-text-to-text model from Wondernutts. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. The card lists the license as apache-2.0.
OpenVINO INT4 AWQ — asymmetric 4-bit weight compression, verified group size 128.
Downloads · 30 days
31
15% of all-time downloads
All-time downloads
206
Public
Repo size
15.7 GB
Likes
0
Public
Click a slice to open those files.
.bin15.7 GB · 100%
From the Hugging Face model README
OpenVINO INT4 AWQ — asymmetric 4-bit weight compression, verified group size 128.
OpenVINO INT4 AWQ deployment conversion of JonathanColetti/Qwen3.8-27B-Uncensored, based on Qwen/Qwen3.8-27B.
Jonathan Coletti performed the original refusal-direction removal and published the BF16 source checkpoint. Qwen created the base model. Wondernutts performed this OpenVINO conversion and controlled deployment validation; this repository does not claim authorship of the upstream model work.
The original plain-INT4 OpenVINO baseline remains untouched at Wondernutts/Qwen3.8-27B-Uncensored-int4-ov.
The uploaded openvino_config.json and conversion manifest record:
JonathanColetti/Qwen3.8-27B-Uncensored@5bb7aa90f0efef548e87005b1fb7658e522b6b7fQwen/Qwen3.8-27B@1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0mtp.* source tensorsThe BF16 source checkpoint contains all 15 verified MTP tensors, restored from the Qwen base model and checked before conversion. However, the standard OpenVINO VLM export used here does not expose a separate MTP draft head. Do not interpret the source-tensor verification as a claim that this OpenVINO package performs MTP speculative decoding.
The repository includes AB_VALIDATED.json, comparing this AWQ candidate against the untouched plain-INT4 Wondernutts baseline. The record reports passed: true; both models produced the expected math answer, coherent roleplay output, and correct red-image recognition.
| CPU DQ0 smoke test | AWQ g128 candidate | Plain INT4 baseline |
|---|---|---|
| Load | 11.78 s | 11.27 s |
| Math generation | 38.67 s | 38.73 s |
| Roleplay generation | 117.26 s | 117.34 s |
| Vision generation | 66.69 s | 66.91 s |
This controlled CPU DYNAMIC_QUANTIZATION_GROUP_SIZE=0 smoke A/B shows no meaningful generation-latency regression in these three checks. It is a functional comparison, not a throughput benchmark: no tokens-per-second claim is made, and the small timing differences should not be generalized beyond this run and configuration.
The validation file also records that the candidate is AWQ while the preserved baseline is not; both use group-128 INT4 storage.
Measured with the established qwen38_arc_bench.py harness on one Intel Arc Pro B70, OpenVINO runtime 5c2085a1f7a8, with thinking disabled. These are results from this specific system and harness, not universal performance claims.
| DQ128 measurement | Result |
|---|---|
| Model load | 16.42 s |
| Short TTFT | 72.95 ms |
| Short decode | 32.727 tok/s |
| Warm PP, 605 input tokens | 2,234.52 tok/s |
| Warm PP, 2,125 input tokens | 2,357.80 tok/s |
| Warm PP, 6,205 input tokens | 2,306.57 tok/s |
| Warm PP, 12,365 input tokens | 2,216.57 tok/s |
| TG at those four prompt lengths | 31.97 / 31.16 / 30.85 / 30.18 tok/s |
| KESTREL-47 retrieval, 11,263 tokens | exact |
| KESTREL-47 TTFT | 4,985.38 ms |
| KESTREL-47 decode | 27.216 tok/s |
| VRAM after suite | 17.25 / 31.89 GB |
| DQ0 measurement | Result |
|---|---|
| Model load | 15.66 s |
| Short decode | 32.789 tok/s |
| Warm PP, 605 input tokens | 1,693.17 tok/s |
| Warm PP, 2,125 input tokens | 1,732.15 tok/s |
| Warm PP, 6,205 input tokens | 1,716.12 tok/s |
| Warm PP, 12,365 input tokens | 1,657.47 tok/s |
| KESTREL-47 retrieval, 11,263 tokens | exact |
| KESTREL-47 TTFT | 6,744.35 ms |
| KESTREL-47 decode | 27.806 tok/s |
| VRAM after suite | 18.22 / 31.89 GB |
Both profiles passed the math and Skyrim roleplay-coherence gates. DQ128 is the recommended serving profile: decode performance is effectively tied, while 12K warm prompt processing is approximately 33.7% faster than DQ0 (2,216.57 versus 1,657.47 tok/s). The earlier CPU DQ0 smoke A/B above remains a functional candidate-versus-baseline check; it should not be conflated with this Arc GPU benchmark.
The source model substantially reduces—but does not eliminate—refusal behavior from Qwen3.8-27B using Heretic. Its source card reports the original methodology, measured behavior, limitations, and safety caveats. Review both the uncensored source card and Qwen base card before deployment.
These links provide conversion/runtime provenance. They are not claims of authorship over the Qwen or JonathanColetti source weights.
import openvino_genai as ov_genai
from huggingface_hub import snapshot_download
model_dir = snapshot_download(
"Wondernutts/Qwen3.8-27B-Uncensored-int4-awq-g128-ov"
)
pipe = ov_genai.VLMPipeline(
model_dir,
"GPU",
DYNAMIC_QUANTIZATION_GROUP_SIZE=128,
)
config = ov_genai.GenerationConfig()
config.max_new_tokens = 512
config.do_sample = True
config.temperature = 1.0
config.top_p = 0.95
config.top_k = 20
result = pipe.generate("Write a short scene in a candlelit inn.", generation_config=config)
print(result)
For direct answers, apply the repository's chat template with thinking disabled where supported. Native image input requires the multimodal VLMPipeline API and a compatible OpenVINO GenAI build.
AutoModelForImageTextToText.