Qwopus3.8-27B-Flash-INT4-W4A16

🪐 Qwopus3.8-27B-Flash (W4A16)

<div align="center"> <img src="https://cdn-uploads.huggingface.co/production/uploads/66309bd090589b7c65950665/GvN4SANiFepBuWcsoWbOX.jpeg" alt="IMG_7359" width="82%"/> </div>

[!NOTE] Qwopus3.8-27B-Flash W4A16 is a 4-bit weight / 16-bit activation quantized version of Qwopus3.8-27B-Flash, using AutoRound with GPTQ group size 128. It preserves the Flash model's efficiency characteristics while reducing memory footprint.

[!TIP] W4A16 Quantization · ~65% Smaller · Same Efficiency Characteristics · FP16 Parity on ARC-Easy

📊 Benchmark Results

ARC-Easy

The W4A16 quantized version matches or slightly exceeds the FP16 original on ARC-Easy.

[!NOTE] Additional benchmarks (MMLU, HumanEval, BBH, WikiText2 PPL) are in progress. Results will be added to this README when complete.

💾 Quantization Details

Model size: ~19GB (vs ~54GB FP16) — 65% reduction

The quantization preserves the following layers in FP16 for quality:

• Linear attention projection matrices (in_proj_a, in_proj_b) across all 64 layers
• MTP (Multi-Token Prediction) module
• Visual encoder

🚀 Usage

Works with any framework supporting GPTQ/AutoRound checkpoints:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Jackrong/Qwopus3.8-27B-Flash-W4A16/Jackrong_Qwopus3.8-27B-Flash-w4g128"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto"
)

Or via vLLM for high-throughput serving:

vllm serve Jackrong/Qwopus3.8-27B-Flash-W4A16/Jackrong_Qwopus3.8-27B-Flash-w4g128 \
    --dtype auto --tensor-parallel-size 1

🎯 Recommended Use Cases

• Local MTP speculative decoding where decoder throughput and draft acceptance matter.
• Long-running agent workflows with repeated tool calls and iterative edits.
• Resource-conscious workloads where wall-clock time and generated-token cost are operational constraints.
• Memory-constrained environments where the 65% size reduction enables local deployment on smaller GPUs.

⚠️ Limitations

• Accuracy–Efficiency Trade-off: The underlying Flash fine-tuning prioritizes practical inference efficiency. On the reported 2,500-question MMLU-Pro mixed set, Qwopus scores 91.28% versus 92.73% for the Qwen3.8 base comparison.
• Reasoning Stability: Although the long-tail behavior is improved in the reported evaluation, edge cases may still exhibit reasoning drift, loops, or inefficient trajectories.
• Agent Dependence: Tool-use quality and task completion remain dependent on the prompt, tool environment, orchestration layer, and feedback loop.
• Experimental Release: This is an independent experimental release for research, local evaluation, and technical exploration.

🙏 Acknowledgements

Special thanks to:

• Qwen for the Qwen3.8-27B base-model foundation.
• Jackrong for the Qwopus3.8-27B-Flash fine-tuning.
• Unsloth for practical, efficient large-model fine-tuning tooling.
• The open-source community for training tools, evaluation methods, and technical discussion.