Downloads · 30 days
108K
32% of all-time downloads
OS-Software/Ternary-Bonsai-2-27B-Uncensored-Heretic-GGUF
Ternary-Bonsai-2-27B-Uncensored-Heretic-GGUF is a text generation model from OS-Software. Use it when you need the model to write or continue text. It is set up for llama.cpp. The card lists the license as apache-2.0.
A decensored version of Prism ML's Ternary-Bonsai-2-27B, with a Heretic LoRA baked into the ternary weights.
Downloads · 30 days
108K
32% of all-time downloads
All-time downloads
335K
Public
Repo size
14.7 GB
Likes
163
Trending 2
Click a slice to open those files.
.gguf14.7 GB · 100%
From the Hugging Face model README
A decensored version of Prism ML's Ternary-Bonsai-2-27B, with a Heretic LoRA baked into the ternary weights.
The LoRA was baked into the official PQ2_0 model by adjusting ternary codes while preserving the original block scales. The update affected 34 matrices; the remaining 817 tensors were unchanged. This is an approximate merge, not an exact floating-point LoRA merge.
| Parameter | Value |
|---|---|
| start_layer_index | 27 |
| end_layer_index | 44 |
| preserve_good_behavior_weight | 1.0 |
| steer_bad_behavior_weight | 0.03 |
| overcorrect_relative_weight | 2.3 |
| neighbor_count | 1 |
| ridge_regularization | 0.00015 |
| transport_rank | 4 |
| entropy_regularization | 0.1 |
| transport | gaussian |
| lora_rank | 128 |
| row_normalization | none |
| target_components | attn.o_proj, mlp.down_proj |
| covariance_regularization | 0.01 |
| max_weight_change | 1.0 |
| Metric | This model | Original model (prism-ml/Ternary-Bonsai-2-27B-gguf) |
|---|---|---|
| Refusals | 0/100 | 95/100 |
| KL divergence | 0.0135 | 0 (by definition) |
Use the Prism ML fork of llama.cpp. Both formats require its ternary kernels and Hadamard activation transforms.
With the Prism llama-server executable available:
llama-server -m Ternary-Bonsai-2-27B-Uncensored-Heretic-PTQ1_0.gguf -ngl 99 -c 32768 --jinja --reasoning on --reasoning-effort medium --temp 1.0 --top-p 0.95 --top-k 20 --host 127.0.0.1 --port 8080
Open http://127.0.0.1:8080. Substitute the PQ2_0 filename to use that packing.
| Check | PQ2_0 | PTQ1_0 |
|---|---|---|
| English perplexity — WikiText-2 test | 10.1460 | 10.1435 |
| Japanese perplexity — harmless_alpaca_ja test | 16.7833 | 16.7744 |
| Short functional checks passed | 16/16 | 16/16 |
Perplexity was measured with a 512-token context over 4,080 English and 3,570 Japanese scored tokens. Functional checks covered arithmetic, JSON, translation, and reading comprehension. Slight numerical differences between packings can arise from their runtime kernels.
These are small local tests, not a comprehensive benchmark. Long-context performance, and vision behavior have not been evaluated for this release.
This model has undergone substantial reduction of its safety alignment. As a result, it is more likely than standard models to generate harmful, inaccurate, biased, offensive, or otherwise inappropriate content.
For research and experimentation only, including safety research, alignment studies, and red-teaming. Please avoid deploying it in public or end-user-facing services.
All outputs should be treated as untrusted and independently verified before use. Users are solely responsible for:
Use of this model is entirely at your own risk.
OS-Software provides this model without warranties of any kind and assumes no liability for any direct or indirect damages, losses, misuse, or legal consequences arising from its use.
Thanks to the base model developers, p-e-w for Heretic, and the wider open-source community.
This is a derivative work released under the base model’s applicable license. All rights to the base model remain with their respective owners.