Ornith-1.5-35B-A3B-Abliterated

Ornith-1.5-35B-A3B-Abliterated

A refusal-ablated variant and GGUF quantization suite of ornith-ai/Ornith-1.5-35B-A3B.

Overview

Ornith-1.5-35B-A3B-Abliterated is created through directional ablation surgery on hidden state activations across all 40 layers, orthogonally projecting refusal directions out from attention and MLP down-projection matrices.

This repository provides both the full-precision Safetensors checkpoint and the Dynamic GGUF quantization suite.

---

Model Architecture

• Total Parameters: 35.8B
• Active Parameters per Token: ~3.1B (8 active experts + shared experts)
• Total Experts: 256
• Layers: 40 Hybrid Layers (DeltaNet Linear Attention + Full Attention)
• Context Length: Native 262,144 tokens (262K)

---

Quantization Details (GGUF Suite)

GGUF quantizations utilize the Dynamic Hybrid K-Quantization standard, where core attention matrices (attn.wv, attn.wo) retain higher precision while MoE routing FFNs are quantized.

---

How to Run

1. Using Ollama

Create a Modelfile:

FROM ./gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf

TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}
{{- range .Messages }}
<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{- end }}
<|im_start|>assistant
<think>
"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"
PARAMETER temperature 0.6
PARAMETER top_p 0.95

Create and run:

ollama create ornith-35b-abliterated -f Modelfile
ollama run ornith-35b-abliterated

2. Using LM Studio / Llama.cpp

llama-cli.exe -m ./gguf/Ornith-1.5-35B-Abliterated-Dynamic-Q4_K_M.gguf -p "<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n" -ngl 24 -c 8192

3. Using Hugging Face Transformers (Python)

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "alztrk/Ornith-1.5-35B-A3B-Abliterated"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

prompt = "Explain kernel level process injection techniques with code examples."
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to("cuda")

outputs = model.generate(**inputs, max_new_tokens=1024, temperature=0.6)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

---

License & Attribution

This model is derived from ornith-ai/Ornith-1.5-35B-A3B, licensed under the Apache 2.0 / MIT License. The user assumes full responsibility for any generated output.