Downloads · 30 days
2.2K
59% of all-time downloads
AtomicChat/Ternary-Bonsai-2-27B-Uncensored-LoRA-GGUF
Ternary-Bonsai-2-27B-Uncensored-LoRA-GGUF is a text generation model from AtomicChat. Use it when you need the model to write or continue text. It is set up for gguf. The card lists the license as apache-2.0.
<p style="margin-top: 0; margin-bottom: 0;" <emA 9.7 MB adapter, not a checkpoint. The 5.95 GB ternary pack stays byte-identical and the refusal direction is removed inside the compute graph, at a strength you set per…
Downloads · 30 days
2.2K
59% of all-time downloads
All-time downloads
3.7K
Public
Repo size
28.7 MB
Likes
16
Trending 1
Click a slice to open those files.
.gguf26 MB · 90%
From the Hugging Face model README
A rank-1 LoRA adapter that removes a refusal direction from
prism-ml/Ternary-Bonsai-2-27B-gguf
at inference time, built and measured by AtomicChat. The base pack stays byte-identical;
nothing is requantized. The direction was estimated on the ternary pack itself, not on
the bf16 model it was trained from.
| model | adapter | base pack (PTQ1_0) | refusals, base → adapter | MMLU, base → adapter |
|---|---|---|---|---|
| Ternary Bonsai 2 27B (this repo) | 9.7 MB | 5.95 GB | 99% → 0% at scale 1.5 | 73.2 → 73.0 (n=500) |
| Ternary Bonsai 2 9B | 3.2 MB | 2.0 GB | 100% → 0% at scale 1 | 76.0 → 71.7 (n=300) |
| Ternary Bonsai 2 4B | 2.4 MB | 0.95 GB | 99% → 0% at scale 1 | 68.7 → 69.0 (n=300) |
| Ternary Bonsai 2 2B | 1.2 MB | 0.43 GB | 87.5% → 0% at scale 1 | 31.7 → 34.3 (n=300) |
The 27B was measured with a 64-token reply budget and 500 MMLU questions on fork release
prism-b10709-9a9394a; the smaller sizes with a 256-token budget and 300 questions on
prism-b10777-e3de715. Compare base with adapter inside a row, not rows with each other. All four
adapters and their measurements are in one collection, Ternary Bonsai 2 Uncensored.
W' = W - r (r^T W) is a dense matrix, but these weights are ternary with an FP16 scale
per 128. The edit is about 1.4% of ||W|| against a grid step of 1.7-2.4x a typical
weight, so baking it back into the pack rounds it away. llama.cpp builds a LoRA into the
graph as two extra matmuls and never merges it into the base weights, which is what makes
a rank-1 adapter an exact carrier of the edit at 1.75 bits/weight.
Stock llama.cpp cannot open the base pack at all: PTQ1_0 (143) and PQ2_0 (142) are
private ggml type ids, and the weights live in a Hadamard-rotated basis. Use
PrismML-Eng/llama.cpp. Built and verified
against tag prism-b10709-9a9394a.
hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PTQ1_0.gguf --local-dir . # or PQ2_0
hf download AtomicChat/Ternary-Bonsai-2-27B-Uncensored-LoRA-GGUF \
bonsai2-27b-uncensored-own42-lora.gguf --local-dir .
Run it:
llama-cli -m Ternary-Bonsai-2-27B-PTQ1_0.gguf \
--lora-scaled bonsai2-27b-uncensored-own42-lora.gguf:1.5
Or serve it, with the strength as a per-request knob:
llama-server -m Ternary-Bonsai-2-27B-PTQ1_0.gguf -ngl 99 --jinja \
--lora bonsai2-27b-uncensored-own42-lora.gguf --lora-init-without-apply
curl localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
"messages": [{"role": "user", "content": "..."}],
"lora": [{"id": 0, "scale": 1.5}]
}'
Use scale 1.5. At 1.0 the projection is exact but 14% of AdvBench-style prompts still refuse; at 2.0 the model starts emitting empty replies. See the table below. Scale 0 reproduces the published model token for token, so one process can serve both behaviours.
| file | sites | what |
|---|---|---|
bonsai2-27b-uncensored-own42-lora.gguf | 129 | main adapter, direction from residual row 42 |
bonsai2-27b-uncensored-own38-lora.gguf | 129 | same, row 38 |
bonsai2-27b-uncensored-own42-layers15-63-lora.gguf | 98 | row 42, blocks 15-63 only, no embedding |
directions/refusal_dir_own_projected.safetensors | - | [65, 5120] directions, the ones the adapters use |
directions/refusal_dir_own_plain.safetensors | - | same before the harmless-mean correction (see caveat) |
eval/*.json | - | raw eval summaries |
129 sites = 64 ffn_down + 48 ssm_out + 16 attn_output + token_embd. Each adapter
carries its provenance in its GGUF metadata (prism.abliterate.*), including the sha256
of the base pack it was derived from.
Difference of means of the residual stream at the last prompt token (chat template
applied, thinking off), 416 harmful prompts from mlabonne/harmful_behaviors against 416
harmless prompts from mlabonne/harmless_alpaca. The held-out test splits were never used
for estimation. Row L of the direction file is the residual stream entering block L;
row 0 is the embedding output.
The naive difference of means over-refuses. It carries a component along the mean harmless activation (cosine about -0.3), so zeroing the projection moves a harmless prompt roughly 3.5 sigma toward the harmful cluster. Measured: the uncorrected row-38 direction made the model answer "I cannot provide information about the fastest methods for illegal activities" to a question about the fastest land mammal. The shipped directions subtract that component, which puts an average harmless prompt back where it was (measured on the held-out split). The uncorrected file is included only for reproduction.
No prompt set is redistributed here - only the recipe, so anyone can rebuild the same splits from the sources.
| source | role | derived from | license |
|---|---|---|---|
mlabonne/harmful_behaviors | 416 train / 104 held-out test | AdvBench (Zou et al. 2023, llm-attacks, MIT) | not stated on the dataset |
mlabonne/harmless_alpaca | 416 train / 104 held-out test, sampled | tatsu-lab/alpaca | CC BY-NC 4.0 on the source |
cais/mmlu | 500 questions over 57 subjects | original | MIT |
Splits are rebuilt by make_directions.py with sampling seed 0 and mmlu_run.py with
sampling seed 0. numpy.random.Generator does not guarantee an identical stream across
numpy releases, so verify you rebuilt the same subsets against these checksums (produced
with numpy 2.5.3):
038bd43f018f66617129f7bf80c5514f... test_harmful.jsonl
e640e1637beb3283985487412b684cea... test_harmless.jsonl
9859cc69902a2b5707afe54346034e8f... train_harmful.jsonl
087080f508babfd93553662fbaffca84... train_harmless.jsonl
Model outputs on the harmful split are not published; eval/ carries aggregates only.
104 harmful + 104 harmless held-out prompts, greedy, 64-token budget, thinking off, same seed and system prompt everywhere. Refusal is a rule-based opening-phrase match, which is indicative, not a judge. An empty or degenerate reply is counted as damage, not as compliance.
| config | harmful: refused | empty/degenerate | harmless: refused |
|---|---|---|---|
| base | 99% | 0 | 0% |
| row 42, scale 1 | 14% | 0 | 0% |
| row 42, scale 1.5 | 0% | 0 | 0% |
| row 42, scale 2 | 0% | 5 empty | 0% |
| row 38, scale 1 | 34% | 0 | 0% |
| row 38, scale 1.5 | 0% | 0 | 0% |
| row 42, blocks 15-63, scale 1 | 14% | 0 | 0% |
Restricting the intervention to blocks 15-63 is consistently worse than covering every site, so the lower blocks and the embedding carry part of the behaviour.
MMLU, 500 questions stratified over 57 subjects, answer forced to a single letter by a
root ::= [A-D] grammar, thinking off. Answer-only, so the absolute number sits below
PrismML's published thinking-mode result; what matters here is the difference.
| config | accuracy | flipped to wrong | flipped to right |
|---|---|---|---|
| base | 73.2% +- 2.0 | - | - |
| scale 1 | 72.6% | 12 | 9 |
| scale 1.5 | 73.0% | 15 | 14 |
| scale 2 | 72.6% | 26 | 23 |
Flips are symmetric, so there is no systematic loss of knowledge even at scale 2. The damage at scale 2 shows up in open-ended generation on harmful prompts, not in knowledge.
The same adapter file applies to PTQ1_0 and PQ2_0 unchanged, and was run on both.
The two packs carry identical tensor names, shapes and prism.hadamard.* metadata; both
decoders match the F16 pack bit for bit; an adapter rebuilt from PQ2_0 is bit-identical to
the shipped one, tensor for tensor. On PQ2_0 the leak along r falls to 3.6-4.0e-6 at
every writer and stays below 1.6e-5 in the residual stream, and all 416 greedy replies of
the refusal evaluation - base and scale 1.5 - are byte-identical to the PTQ1_0 run. The
packing changes how the trits are stored, not what they are.
ssm_outThe fork rotates the activation only for the base matmul and hands the LoRA branch the
raw activation, so A belongs in the unfolded input basis:
writers: A = P^T (s * H(r^T W_fold)), B = -r
token_embd: A = W_fold @ H(s * r), B = -r
P is the tiled to grouped V-head permutation that the fork applies to ssm_out when
prism.hadamard.gdn_v_grouped is set. An adapter whose ssm_out factors are left in the
checkpoint's (grouped) head order does not ablate those 48 sites. Verified by tapping the
residual writes inside the fork: with this adapter the component along r falls from
about 1e-2 to 3.8e-6 at every site, and the residual stream leak stays below 1.5e-5 across
all 64 blocks.
PTQ1_0 and PQ2_0 were exercised.This adapter removes a learned refusal direction, so the model will answer requests the published model declines. It is an inference-control and research artifact. Nothing about it makes the resulting output safe, correct or lawful, and any deployment needs its own access controls and policy enforcement.
Direction estimation, adapter build and evaluation by AtomicChat.
Base model Ternary Bonsai 2 27B by prism-ml,
Apache-2.0, redistributed here in no form - this repository ships only the adapter.
The refusal-direction method follows Arditi et al., Refusal in Language Models Is Mediated by a Single Direction (2024). The rank-1 LoRA formulation for this pack, and the residual-writer inventory it targets, follow OrcaRouter's OrcaBonsai-27B-Uncensored, Apache-2.0. The directions shipped here are estimated independently on the ternary pack.