Downloads · 30 days
0
rockerBOO/DreamFast-gemma-3-12b-it-heretic-v2-rank64-lora
DreamFast-gemma-3-12b-it-heretic-v2-rank64-lora is a text generation model from rockerBOO. Use it when you need the model to write or continue text. It is set up for peft.
A rank-64 LoRA extracted via SVD from the weight difference between:
Downloads · 30 days
0
Access
Public
Updated Aug 2, 2026
Repo size
66.4 MB
Likes
0
Public
Click a slice to open those files.
.safetensors66.4 MB · 100%
From the Hugging Face model README
A rank-64 LoRA extracted via SVD from the weight difference between:
Applying this LoRA to the base model at strength 1.0 approximately reproduces the effect of the heretic (refusal-ablation) fine-tune, without needing to distribute or download the full ~24GB fine-tuned checkpoint.
A rank-16 version of this same extraction is also available if you want a smaller file.
For each matching attention/MLP projection weight, the tensor difference tuned - base was computed and factored via SVD into low-rank up/down matrices, truncated to rank 64.
o_proj, down_projq_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj across all transformer blocks) — the heretic fine-tune only meaningfully changed o_proj and down_proj weights, consistent with a targeted refusal-direction ablation rather than a broad fine-tune.adapter_model.safetensors + adapter_config.json — standard PEFT LoRA adapterkohya/gemma3_heretic_diff_r64.safetensors — the same weights in flat kohya-style naming (<module>.lora_up.weight / <module>.lora_down.weight) for tooling that expects that convention instead of PEFTfrom transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_id = "google/gemma-3-12b-it"
model = AutoModelForCausalLM.from_pretrained(base_model_id, dtype="bfloat16", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(model, "rockerBOO/gemma-3-12b-it-heretic-v2-rank64-lora")
Inherits the Gemma license terms from the base model, google/gemma-3-12b-it. See that model's page for the full license text and usage restrictions.