Downloads · 30 days
51
0% of all-time downloads
Freenixi/Abiray-Qwen3.6-27B-NVFP4-GGUF
Abiray-Qwen3.6-27B-NVFP4-GGUF is a image-text-to-text model from Freenixi. Use it for the image-text-to-text task on the model card, and read the license before you ship it in a product. It is set up for gguf. The card lists the license as apache-2.0.
This repository provides a GGUF-converted version of an NVFP4 checkpoint for local inference in LM Studio on Windows, optimized for NVIDIA RTX 5090 / Blackwell systems.
Downloads · 30 days
51
0% of all-time downloads
All-time downloads
16.5K
Public
Repo size
19.7 GB
Likes
19
Public
Click a slice to open those files.
.gguf19.7 GB · 100%
From the Hugging Face model README
This repository provides a GGUF-converted version of an NVFP4 checkpoint for local inference in LM Studio on Windows, optimized for NVIDIA RTX 5090 / Blackwell systems.
This release is a mixed-source packaging:
This is not an official upstream release. It is a community conversion intended for experimentation and local deployment.
| Component | Source |
|---|---|
| Original base model | Qwen/Qwen3.6-27B |
| NVFP4 checkpoint | Abiray/Qwen3.6-27B-NVFP4 |
Vision projector (mmproj-BF16.gguf) | unsloth/Qwen3.6-27B-GGUF |
| GGUF conversion tooling | llama.cpp |
| NVFP4 conversion branch used | pull/21095/head:pr-21095 |
Typical structure:
Qwen3.6-27B-NVFP4.gguf — main model GGUFmmproj-BF16.gguf — vision projector, required for image inputVision functionality requires both files. If mmproj-BF16.gguf is missing, image input will not work in LM Studio.
Tested environment:
llama.cpp PR branch pr-21095LM Studio added explicit Qwen 3.6 support starting from version 0.4.12.
Observed performance on RTX 5090:
Performance may vary depending on prompt length, context usage, GPU offload settings, driver version, LM Studio runtime version, and workload type.
mmproj-BF16.ggufQwen3.6-27B enables reasoning by default and may emit reasoning content in <think>...</think> blocks.
Disabling reasoning is not done via /think or /nothink.
Where supported, use chat template configuration such as:
chat_template_kwargs = {"enable_thinking": False}
OR, start the server as:
.\build\bin\Release\llama-server.exe --% -m .\AxionML-Qwen3.5-9B-NVFP4-GGUF\axionml-qwen3.5-9b-nvfp4.gguf --host 127.0.0.1 --port 8080 -c 96000 -ngl 99 --chat-template-kwargs "{\"enable_thinking\":false}"
LM Studio may not expose a visible reasoning toggle for custom GGUF imports, depending on how the model metadata is recognized.
The vision projector file was sourced from:
For LM Studio vision support, place the main model GGUF and mmproj-BF16.gguf together in the imported model folder. Image input should become available when LM Studio recognizes the model as a vision-language model.
The main GGUF was converted from:
The conversion used a development branch of llama.cpp that includes work for compressed-tensors NVFP4 conversion:
Sanitized workflow:
cd .\llama.cpp
git fetch origin pull/21095/head:pr-21095
git checkout pr-21095
python -m pip install --no-cache-dir -r requirements\requirements-convert_hf_to_gguf.txt
The upstream tokenizer configuration required a compatibility patch before conversion:
(Get-Content <model_dir>\tokenizer_config.json) `
-replace '"tokenizer_class": "TokenizersBackend"', '"tokenizer_class": "Qwen2Tokenizer"' `
| Set-Content <model_dir>\tokenizer_config.json
Then the model was converted using:
python convert_hf_to_gguf.py <model_dir> --outfile <output.gguf> --outtype bf16 --verbose
pr-21095 llama.cpp branch rather than ordinary master at the time of conversion.--outtype bf16 conversion path does not mean the entire release is a normal BF16 model.Qwen3.6-27B-NVFP4.ggufmmproj-BF16.ggufThis repository is a derivative conversion based on upstream model assets.
Please review and follow the licenses and usage terms of:
Users are responsible for ensuring compliance with all upstream model terms.
This release is intended for:
It is not presented as an official or production-grade distribution.