Downloads · 30 days
29
34% of all-time downloads
Vita0818/Vireqo-27B-260815
Vireqo-27B-260815 is a text generation model from Vita0818. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
Experimental, unofficial, text-only, Bonsai-guided extreme quantization of Qwen3.8-27B.
Downloads · 30 days
29
34% of all-time downloads
All-time downloads
85
Public
Repo size
4.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.9 GB · 100%
From the Hugging Face model README
Experimental, unofficial, text-only, Bonsai-guided extreme quantization of Qwen3.8-27B.
Vireqo-27B-260816 is a 4.54 GiB GGUF experiment built from both Qwen/Qwen3.8-27B and prism-ml/Bonsai-27B-gguf. It is deliberately optimized for one narrow objective: fit a 27B-class text model into a very small memory budget and retain enough language behavior to produce short, intelligible replies on an Apple Silicon Mac.
This is not a conventional post-training quantization of Qwen3.8-27B, is not an official Qwen or Prism ML release, and must not be described as a pure or quality-preserving Qwen3.8 Q1 conversion. The main Q1 matrices retain Bonsai's binary sign patterns; selected scales and all protected tensors inject information from Qwen3.8-27B. The exact construction is disclosed below.
中文摘要:Vireqo-27B-260816 是一个非官方、纯文本、以 Q1 为主的极限实验模型。它同时使用 Qwen3.8-27B 的权重信息与 Bonsai-27B 的 Q1 二值符号骨架。实测能在目标 Mac 上加载并生成可理解的中英文短回复,但长输出容易复读、漂移或泄漏 </think>,不能当作正常质量的 27B 模型。
| Item | Value |
|---|---|
| File | Vireqo-27B-260816.gguf |
| Size | 4,876,196,672 bytes |
| Binary size | approximately 4.54 GiB |
| Headroom below 5,000,000,000 bytes | 123,803,328 bytes |
| SHA-256 | 90dce39da7ddff50f28e3e2cf04c6552d968be9e4eee7d094e849620d691a25b |
| Tensor count | 851 |
| Total tensor elements | 26,895,998,464 |
| Whole-file average | approximately 1.4504 bits per element |
| Scope | text only; no vision projector, MTP, or drafter |
The GGUF embeds general.name = Vireqo-27B-260816, the downloaded Qwen3.8 tokenizer and merges, and the Qwen3.8 chat template used during validation.
The released file uses one fixed mixed-precision layout:
| GGUF type | Tensors | Elements | Payload bytes | Source and method |
|---|---|---|---|---|
Q1_0 | 496 | 24,350,556,160 | 3,424,296,960 | Bonsai Q1 signs unchanged; blended Bonsai/Qwen block scales |
Q4_0 | 2 | 2,542,796,800 | 1,430,323,200 | Qwen3.8 token_embd.weight and output.weight |
F32 | 353 | 2,645,504 | 10,582,016 | Qwen3.8 small, 1D, norm, bias, and control tensors after pinned llama.cpp transforms |
For every 128-weight Q1 block in the 496 final Q1 matrices:
s_final = s_bonsai # zero sign flips
d_fit = max(dot(W_qwen, s_bonsai) / 128, 0)
d_final = 0.75 * d_bonsai + 0.25 * d_fit
W_q1 = d_final * s_final
In plain language:
Q1_0 blocks;Q4_0;mmproj, MTP, and external drafter weights are excluded.This makes the artifact Bonsai-guided and Qwen3.8-derived, not a model whose entire parameter set can be attributed to only one source.
Before the final build, a same-route staging experiment used pure Qwen-fitted scales (d_final = d_fit) while still preserving Bonsai signs. It was structurally valid but failed behaviorally: normal chat terminated immediately with EOS, while forced continuation produced repetitive garbage.
The fitted scale was systematically smaller than the original Bonsai scale on average (approximately 0.00906 versus 0.01037). The working interpretation was that the pure fit reduced signal energy cumulatively through the network. Returning to the conservative 0.75 × Bonsai + 0.25 × Qwen-fit blend restored short intelligible replies. The failed staging file is not included in this repository.
The build used pinned inputs rather than floating repository heads:
| Source | Revision | Role |
|---|---|---|
| Qwen/Qwen3.8-27B | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 | BF16 target weights, tokenizer, chat template, Q4 embedding/head, F32 tensors, fitted Q1 scales |
| prism-ml/Bonsai-27B-gguf | f10afb355f104535e3e3e98cf7ab7795c72bd292 | aligned Q1 binary sign prior and anchor scales |
| ggml-org/llama.cpp | ad1de39e0708e3ced9c71bb3c82d93a2c046a73f | conversion transforms and validated runtime |
The exact Bonsai input was Bonsai-27B-Q1_0.gguf, 3,803,452,480 bytes, SHA-256 17ef842e47450caeb8eaa3ebfbbab5d2f2278b62b79be107985fb69a2f819aa0.
The two models were not assumed to be interchangeable merely because shapes matched. Before writing the full artifact, a streaming alignment audit checked all text tensors and sampled Q1 blocks across every candidate large matrix.
These results justified attempting the transfer. They do not establish Qwen3.8 quality retention.
A separate full-file validation pass reported PASS after the final metadata/name rebuild:
The validator proves that the file faithfully implements the declared packing and transfer algorithm. It does not substitute for capability evaluation.
The final tensor payload was tested on an Apple M5 MacBook Pro with a 12 GB task budget, using the pinned llama.cpp commit, full Metal offload, context 1024, flash attention, Q8 K/V cache, thinking disabled, temperature 0, and short generation limits.
| Prompt/diagnostic | Observed result |
|---|---|
| Chinese factual question: capital of France | 法国首都是巴黎。 |
| English exact phrase | I can speak. |
| Chinese short status reply | 你好。我能正常工作。 |
| Three-step tea instruction | Began with intelligible Chinese steps, then exceeded the requested count and repeated |
| Two-turn “蓝鲸” memory test | Recalled 蓝鲸。 first, then repeated and leaked </think> during continued generation |
The exact final named artifact (SHA-256 90dce39d…a25b) was loaded again after its metadata rebuild and answered 法国首都是巴黎。. That run reached 5,273,616,384 bytes maximum RSS (approximately 4.91 GiB), generated at 18.1 tokens/s, and reported zero swaps. An earlier extended five-test suite using the same quantization route, tensor layout, tokenizer, and chat template reached approximately 5.02 GiB and 15.9–17.3 tokens/s. Speed was not an acceptance criterion; the separation between the extended suite and the exact final-file retest is recorded in runtime-validation.json.
Start conservatively:
The following reflects the known-good local configuration. Adjust the binary and model paths for your system.
GGML_METAL_TENSOR_DISABLE=1 ./llama-cli \
-m Vireqo-27B-260816.gguf \
-ngl all -c 1024 -fa on -ctk q8_0 -ctv q8_0 \
--no-warmup --jinja \
--chat-template-kwargs '{"enable_thinking":false}' \
--reasoning off --temp 0 -n 32 -st \
-p '请用一句完整的中文回答:法国的首都是哪里?'
GGML_METAL_TENSOR_DISABLE=1 was required on the tested M5 setup to avoid a Metal Tensor API failure. It is a machine/runtime workaround, not a property required on every Mac. A llama.cpp build must support the model architecture and Q1_0; an older build reporting an unknown tensor type should be updated before the GGUF is considered corrupt.
For a local file, import it with an explicit non-moving mode. The hard-link form consumes almost no additional disk space when the source and LM Studio library are on the same filesystem:
lms import /Users/vita/Models/Vireqo-27B-260816/Vireqo-27B-260816.gguf \
--hard-link \
--user-repo local/Vireqo-27B-260816
Then open Chat, choose Vireqo-27B-260816 in the model loader, and begin with context 1024, maximum GPU offload, flash attention, and a 32–64-token response limit. Use LM Studio's newest installed llama.cpp runtime. See LM-STUDIO-使用指南.md in this repository for import choices, GUI steps, CLI loading, and troubleshooting.
Local compatibility was checked directly with LM Studio's installed llama.cpp-mac-arm64-apple-metal-advsimd-2.28.2 backend: the exact final SHA-256 file reached model loaded with context 1024, full Metal offload, flash attention, and Q8 K/V cache. The temporary loopback server was stopped after the check, and this test did not import or copy the file into the LM Studio library.
This model reached the project's minimum goal—“under decimal 5 GB and able to talk”—but it is severely lossy.
</think> tags.When the full repository is uploaded, the accompanying files document the build rather than asking users to trust this summary:
Vireqo-27B-260816.gguf.provenance.json — per-tensor conversion statistics and method metadata;SHA256SUMS.txt — released artifact checksum;source-revisions.json — source repositories, pinned revisions, and input checksum;quant-map.json — tensor-type map and Q1 scale formula;tensor-stats.json — file, payload, element, and average-bit statistics;gate0-alignment.md / .json — pre-conversion transfer audit;artifact-validation.md / .json — post-build structural and byte-level checks;runtime-validation.json and smoke-test-results.md — local runtime observations;TECHNICAL_README.md — concise technical handoff;LM-STUDIO-使用指南.md — local LM Studio instructions.The distributed artifact is provided under Apache License 2.0, consistent with the source repositories inspected at the pinned revisions. Review LICENSE and NOTICE before redistribution, and re-check the upstream repositories if you build from a different revision.
Vireqo-27B-260816 is the name of this independent experimental artifact. The project is not affiliated with, sponsored by, or endorsed by Qwen, Alibaba Cloud, or Prism ML. “Qwen” and “Bonsai” are used here only to identify the disclosed upstream sources and technical lineage.
If you share results from this artifact, please cite the two upstream model repositories and identify this exact build by name and checksum. There is no paper citation for Vireqo-27B-260816 at this time.