Qwen3.8-27B-q2ks-AutoRound

Model Details

This model is a gguf q4km format of Qwen/Qwen3.8-27B generated by intel/auto-round algorithm using best recipe. Please refer to Section Generate the model for more details.

Please follow the license of the original model.

How To Use

Llama.cpp inference.

wget https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg

./llama-cli -m Intel/Qwen3.8-27B-q2ks-AutoRound/Qwen3.8-27B-Q2_K_S.gguf  -mm Intel/Qwen3.8-27B-q2ks-AutoRound/mmproj-model.gguf

> /image demo.jpeg

Loaded media from 'demo.jpeg'

> Describe this image.
The user wants a description of this image. Let me look carefully at all the elements.

Scene: A beach at what appears to be sunset or sunrise—the sky is bright, washed out, with warm golden light coming from the right side (sun is low, creating a lens flare / warm glow on the woman's hair). The ocean is in the background with a small wave breaking.

Foreground: sandy beach, textured with ripples and footprints.

Subjects: A woman and a dog.

The dog: a light-colored (golden/cream) Labrador or similar large breed, sitting on the sand, facing right toward the woman. It's wearing a colorful patterned harness (blue/teal with red and green dots) and a red leash that's lying on the sand. The dog has one front paw raised and is "shaking hands" with the woman—its paw is resting on her hand. The dog's expression looks happy/attentive, mouth slightly open.

The woman: sitting on the sand, barefoot, wearing a plaid/checkered shirt (dark grey/black and white), dark jeans rolled up. She has long dark hair with a warm golden rim-light from the sun. She's smiling, looking at the dog. She has a white wristband/watch on her left wrist. Her right hand is reaching toward the dog's paw (the handshake), her left hand is raised near her chest, perhaps holding something small or just gesturing.

Mood: warm, affectionate, peaceful, golden hour.

Composition: horizontal/landscape, subjects in the lower-middle, large sky above.

I'll write a clear, detailed description. I'll keep it organized and natural, not too long. I'll mention the handshake gesture as the focal point, the lighting, the setting, and the clothing. I'll avoid over-interpreting but note the evident affection.

Let me make sure I describe accurately: the dog's raised paw is on the woman's outstretched hand—yes, that's a "shake" pose. The leash is red, coiled on the sand near the dog's feet. The harness is patterned.

I'll write the final description now.

[End thinking]

This is a warm, golden-hour photograph taken on a sandy beach, with the ocean and a gently breaking wave visible in the background beneath a bright, pale sky. The low sun (coming from the right) casts a soft, golden glow across the scene and rims the woman's hair with light.

In the center of the image, a woman and a dog are sitting on the sand, facing each other in a moment of affection. The dog—a large, cream-colored Labrador-type breed—is seated and wearing a colorful patterned harness (teal with red and green dots) attached to a red leash that lies coiled on the sand. It has one front paw raised and extended, resting on the woman's outstretched hand in a "shake" pose, its head turned toward her with an alert, happy expression.

The woman is seated barefoot on the sand, wearing a dark plaid (checkered) shirt and dark jeans rolled at the ankles, with a white band on her wrist. She has long dark hair and is smiling as she looks at the dog, one hand meeting the dog's paw and the other raised near her chest.

Overall, the image conveys a peaceful, affectionate bond between a person and their dog, captured in soft, warm light at the edge of the sea.

Generate the model

Here is the sample command to reproduce the model

auto_round_best Qwen/Qwen3.8-27B  --format "gguf:q2_k_s" --output_dir tmp_autoround --enable_alg_ext

Ethical Considerations and Limitations

The model can produce factually incorrect output, and should not be relied on to produce factually accurate information. Because of the limitations of the pretrained model and the finetuning datasets, it is possible that this model could generate lewd, biased or otherwise offensive outputs.

Therefore, before deploying any applications of the model, developers should perform safety testing.

Disclaimer

The license on this model does not constitute legal advice. We are not responsible for the actions of third parties who use this model. Please consult an attorney before using this model for commercial purposes.

Citation

@article{cheng2025signroundv2,
  title={SignRoundV2: Toward Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs},
  author={Cheng, Wenhua and Zhang, Weiwei and Guo, Heng and Shen, Haihao and Ma, Zaner},
  journal={arXiv preprint arXiv:2512.04746},
  year={2025}
}