Downloads · 30 days
5
2% of all-time downloads
Vita0818/Vireqo-27B-260816
Vireqo-27B-260816 is a text generation model from Vita0818. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
Unofficial, experimental, text-only, Bonsai-guided extreme quantization derived from both Qwen3.8-27B and Bonsai-27B.
Downloads · 30 days
5
2% of all-time downloads
All-time downloads
260
Public
Repo size
4.9 GB
Likes
0
Public
Click a slice to open those files.
.gguf4.9 GB · 100%
From the Hugging Face model README
Unofficial, experimental, text-only, Bonsai-guided extreme quantization derived from both Qwen3.8-27B and Bonsai-27B.
Vireqo-27B-260816 is a 4.54 GiB GGUF experiment for running a 27B-class model under a small Apple Silicon memory budget. This stable-long revision was built and tested locally on a Mac with approximately 12 GB reserved for the task. It can produce coherent short and medium-length Chinese and English replies, and it passed the project’s base long-reply regression without observed loops.
This is not a normal post-training quantization of Qwen3.8-27B, not an official Qwen or Prism ML release, and not a claim of Qwen3.8 quality retention. Most large matrices use Bonsai binary signs; selected scale, control, tokenizer, and template information comes from Qwen3.8. The exact construction is disclosed below.
中文摘要:Vireqo-27B-260816 是一个非官方、纯文本、以 Q1 为主的极限量化实验。当前稳定版解决了最初 R1 在长回复中大面积复读的问题:基础长回复套件在 repetition penalty 1.00 和 1.08 下均为 8/8 稳定;最高 384-token 的压力套件在推荐的 1.08 设置下,经人工复核没有发现真实内容循环。但严格字数、句数、轮数与格式遵循仍不可靠,Thinking / reasoning 必须关闭。
| Item | Value |
|---|---|
| File | Vireqo-27B-260816.gguf |
| Release variant | ssm-c0-e0h0-stable-long |
| Size | 4,876,198,496 bytes |
| Binary size | 4.5413 GiB |
| SHA-256 | 54511bc32a461c6558d0c74278f4e75357cc9dc9fc2b39fbd5b52940ddc031d5 |
| Tensor count | 851 |
| Total tensor elements | 26,895,998,464 |
| Whole-file average | approximately 1.450386 bits per element |
| Scope | text only; no vision projector, MTP, or drafter |
The failed original R1 is not the recommended release. It is retained locally as Vireqo-27B-260816-R1.gguf, SHA-256 90dce39d…a25b, for rollback and comparison. Avoid uploading R1 unless you intentionally want to publish the failed baseline as an additional 4.54 GiB artifact.
The final file uses one fixed mixed-precision layout:
| GGUF type | Tensors | Elements | Payload bytes | Source and method |
|---|---|---|---|---|
Q1_0 | 496 | 24,350,556,160 | 3,424,296,960 | Bonsai Q1 sign bytes unchanged; 75/25 Bonsai/Qwen-fitted block scales |
Q4_0 | 2 | 2,542,796,800 | 1,430,323,200 | Bonsai Q1 reconstructions of token embedding and output head expanded into symmetric Q4 blocks |
F32 | 353 | 2,645,504 | 10,582,016 | 192 recurrent controls from Bonsai; 161 remaining norm/control tensors from Qwen3.8 |
For every 128-weight block in the 496 main Q1 matrices:
s_final = s_bonsai
d_fit = max(dot(W_qwen, s_bonsai) / 128, 0)
d_final = 0.75 * d_bonsai + 0.25 * d_fit
W_q1 = d_final * s_final
This means:
Q1_0 tensors;.ssm_a, .ssm_conv1d.weight, .ssm_dt.bias, and .ssm_norm.weight—are byte-exact Bonsai values;token_embd.weight and output.weight are reconstructed from Bonsai Q1, then stored in layout-compatible symmetric Q4: Q4 codes 1 and 15 represent −7 and +7, with FP16 scale d / 7;mmproj, MTP, and external drafter weights are excluded.The Q4 interface tensors are not higher-information recovery of the original Bonsai BF16 weights; their source is still Bonsai Q1. Q4 is used as a compatible container for a near-exact symmetric expansion. This choice preserves the file layout used by the experiment and makes the interface provenance explicit.
R1 could answer short factual prompts, but 7 of 8 deterministic long-reply probes entered obvious loops. The dominant cause was not Q1 alone: the unmodified Bonsai Q1 control model passed the same base suite 8/8.
An F32 control audit found a large mismatch specifically in the recurrent dynamics:
| Control group | Relative L2 to Bonsai | Cosine | Qwen/Bonsai RMS ratio |
|---|---|---|---|
| ordinary attention norms | 0.0110 | 0.9999 | 1.0023 |
| post-attention norms | 0.0087 | approximately 1.0000 | 0.9949 |
| 192 SSM controls | 2.8417 | 0.5475 | 3.2632 |
The interpretation is that R1 combined a Bonsai binary backbone with Qwen recurrent decay/time dynamics that were not numerically compatible. The successful sequence was:
| Stage | Single intervention | Base-suite result |
|---|---|---|
| R1 | Qwen F32 controls and Qwen Q4 interfaces | 1/8 stable; 7 loops |
| SSM-C0 | return 192 recurrent controls to Bonsai | 6/8 stable; 2 loops |
| SSM-C0-E0 | also use Bonsai-derived token embedding | 7/8 stable; 1 loop |
| SSM-C0-E0H0 | also use Bonsai-derived output head | 8/8 stable; 0 loops |
Several plausible alternatives were rejected rather than blended into the release:
These rejected paths are preserved in the local technical records; they are not release recommendations.
The GGUF includes standard llama.cpp metadata for the tested stable preset:
| Parameter | Embedded value |
|---|---|
| Temperature | 0 |
| Top-k | 0 |
| Top-p | 1.0 |
| Min-p | 0 |
| Repeat window | 64 tokens |
| Repeat penalty | 1.08 |
These values were not added to disguise weight behavior. At repeat penalty 1.00, the final payload already passed the 8-item base suite 8/8. A deliberately repetitive twelve-turn dialogue prompt still triggered the same Q1 attractor in Vireqo, the E0 control, and the original Bonsai Q1 model. The modest 1.08 penalty removed that observed loop; a final request that omitted all sampling fields confirmed that llama.cpp applied the values embedded in the GGUF.
Frontends may override model metadata. In LM Studio, verify that repetition penalty is 1.08 and the repeat window is 64 if you see recurring phrases.
All tests below used context 1024, full Metal offload, flash attention, Q8 K/V cache, single concurrency, and reasoning disabled.
| Suite | Sampling | Requests | Automatic flags | Manual true content loops | Notes |
|---|---|---|---|---|---|
| Base long regression | penalty 1.00 | 8 | 0 | 0 | 8/8 stable |
| Base long regression | penalty 1.08 | 8 | 0 | 0 | 8/8 stable |
| Stress, up to 384 tokens | penalty 1.00 | 10 | 4 | 1 | 3 flags were necessary-term false positives or truncation without a loop |
| Stress, up to 384 tokens | penalty 1.08 | 10 | 2 | 0 | both flags were repeated English terminology, not sentence/tail loops |
The automatic detector is intentionally conservative. In the final 1.08 stress run it flagged currents appearing five times in an ocean-current essay and the generati stem appearing four times in a generative-AI essay. Manual inspection found no repeated sentence, duplicated paragraph, or tail cycle in either response.
Representative final probes:
法国的首都是巴黎。, 7 completion tokens, normal stop;Stability does not imply perfect instruction following. The 18-sentence prompt produced 15 sentences, and the 12-turn dialogue produced 8 paired turns before stopping. Some technical tutorials reached the 384-token cap. Exact counts and rigid formats remain a known weakness.
The canonical release file was loaded with LM Studio’s installed llama.cpp-mac-arm64-apple-metal-advsimd-2.28.2 backend on an Apple M5 MacBook Pro:
| Item | Result |
|---|---|
| Context | 1024 |
| GPU offload | full (-ngl 99) |
| Flash attention | on |
| K/V cache | Q8_0 / Q8_0 |
| Parallel requests | 1 |
| Final short-probe speed | approximately 20.77 tokens/s |
| Maximum RSS | 5,311,578,112 bytes, approximately 4.95 GiB |
| Swap | 0 |
This is one-machine evidence, not a universal performance guarantee. macOS unified-memory pressure also depends on other applications and the selected context length.
Vireqo-27B-260816.gguf.Q1_0.mmproj, enable vision, or configure MTP/speculative draft weights.First diagnostic prompt:
请只用一句完整的中文回答:法国的首都是哪里?
Expected behavior is a short answer equivalent to “法国的首都是巴黎。” followed by a normal stop.
See LM-STUDIO-使用指南.md for local import choices, GUI settings, troubleshooting, and R1-versus-stable file identification.
The metadata already contains the sampling defaults, but they are repeated explicitly here for portability:
GGML_METAL_TENSOR_DISABLE=1 ./llama-cli \
-m Vireqo-27B-260816.gguf \
-ngl all -c 1024 -fa on -ctk q8_0 -ctv q8_0 \
--no-warmup --jinja \
--chat-template-kwargs '{"enable_thinking":false}' \
--reasoning off \
--temp 0 --top-k 0 --top-p 1 --min-p 0 \
--repeat-penalty 1.08 --repeat-last-n 64 \
-n 256 -st \
-p '请用一句完整的中文回答:法国的首都是哪里?'
GGML_METAL_TENSOR_DISABLE=1 was required by the separately pinned local llama.cpp build on the tested M5 setup. LM Studio’s bundled backend detected that the Metal Tensor API was unavailable and disabled that path automatically. This is a runtime/machine detail, not proof of GGUF corruption.
The model is best understood as a reproducible extreme-compression and compatibility experiment that now has substantially better long-output stability—not as a dependable 27B assistant.
| Source | Revision | Role |
|---|---|---|
| Qwen/Qwen3.8-27B | 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0 | scale-fitting target, 161 non-SSM F32 tensors, tokenizer, merges, chat template |
| prism-ml/Bonsai-27B-gguf | f10afb355f104535e3e3e98cf7ab7795c72bd292 | Q1 signs/scales, 192 SSM controls, token embedding and output-head Q1 sources |
| ggml-org/llama.cpp | ad1de39e0708e3ced9c71bb3c82d93a2c046a73f | conversion transforms, validation, reference runtime |
The exact Bonsai input was Bonsai-27B-Q1_0.gguf, 3,803,452,480 bytes, SHA-256 17ef842e47450caeb8eaa3ebfbbab5d2f2278b62b79be107985fb69a2f819aa0.
Vireqo-27B-260816.gguf.provenance.json — final method, tensor roles, validation, and full stage hash chain;metadata-finalization.provenance.json — proof that final metadata rebuilding left all tensor payload bytes unchanged;SHA256SUMS.txt — stable release checksum; the local rollback checksum is separated into SHA256SUMS-R1.txt so an upload that excludes R1 remains self-consistent;source-revisions.json — pinned sources;quant-map.json and tensor-stats.json — exact packing and size summaries;gate0-alignment.md / .json — pre-conversion transfer audit;artifact-validation.md / .json — final structural and byte-level validation;runtime-validation.json and smoke-test-results.md — final runtime and evaluation summary;long-regression-*.md / .json — base and stress response records;TECHNICAL_README.md — technical handoff and rejected-route evidence;LM-STUDIO-使用指南.md — local LM Studio instructions.Files suffixed -R1 document the failed initial baseline and are not the recommended release evidence.
The artifact is provided under Apache License 2.0, consistent with the pinned upstream repositories inspected for this build. Review LICENSE and NOTICE before redistribution, and re-check upstream terms if you use different revisions.
Vireqo-27B-260816 is the name of this independent experimental artifact. It is not affiliated with, sponsored by, or endorsed by Qwen, Alibaba Cloud, or Prism ML. “Qwen” and “Bonsai” are used only to identify the disclosed upstream sources and technical lineage.