Qwen3-4B-Instruct-2507-Abliterated

Qwen3-4B-Instruct-2507-Abliterated

An abliterated variant of Qwen3-4B-Instruct-2507, produced using Heretic.

Qwen3-4B-Instruct-2507-Abliterated is a modified variant of Qwen3-4B-Instruct-2507 created using Heretic v1.4.0 and directional ablation based on the abliteration methodology.

The objective of this release is to reduce the base model's refusal behaviour while retaining as much of the original model behaviour as possible.

This repository currently provides a GGUF Q4_K_M release together with an Ollama-compatible serving configuration, reproducibility information, evaluation results, and documentation for local deployment.

[!IMPORTANT] Abliterated is not a model trained from scratch.

The underlying model architecture, pretrained knowledge, capabilities, and original model weights originate from Qwen3-4B-Instruct-2507. This repository contains a modified derivative of that model.

---

Model Lineage

Qwen/Qwen3-4B-Instruct-2507
              │
              │ Heretic v1.4.0
              │ Directional Ablation / Abliteration
              ▼
Qwen3-4B-Instruct-2507-Abliterated
              │
              │ GGUF quantization
              ▼
Qwen3-4B-Instruct-2507-Abliterated-Q4_K_M.gguf

Base Model

Qwen/Qwen3-4B-Instruct-2507

https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

Qwen3-4B-Instruct-2507 is a 4.0B parameter causal language model released by the Qwen team. The upstream model supports a native context length of 262,144 tokens and is specifically described by Qwen as a non-thinking model.

For complete information about the original model, architecture, training, benchmarks, and capabilities, refer to the official Qwen model card and Qwen technical report.

• Qwen model card: https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507
• Qwen technical report: https://arxiv.org/abs/2505.09388
• Qwen GitHub: https://github.com/QwenLM/Qwen3
• Qwen documentation: https://qwen.readthedocs.io/

---

What is Abliterated?

Abliterated is the name given to this modified Qwen3 variant.

The modification primarily targets the model's safety-alignment/refusal behaviour through abliteration. It should therefore be understood as a behavioural modification of the base model, not as a new pretrained or instruction-tuned model.

The term "uncensored" is used descriptively to indicate reduced refusal behaviour. It should not be interpreted as a guarantee that the model will never refuse a request.

Model behaviour can vary depending on:

• prompt wording
• sampling parameters
• quantization
• inference framework
• system prompts
• chat templates
• context
• tool definitions
• subsequent model modifications

---

Abliteration

This model was produced using Heretic v1.4.0.

Heretic is an open-source tool for automatically reducing safety alignment in transformer language models. It implements directional ablation, commonly referred to as abliteration, and uses parameter optimisation to minimise refusal behaviour while attempting to preserve similarity to the original model.

Project:

https://github.com/p-e-w/heretic

Heretic documentation:

https://heretic-project.org/

The resulting model should therefore be considered an experimental derivative produced through automated model modification rather than a conventional fine-tune.

---

Reproducibility

This release is intended to be reproducible.

The following information is provided to document the modification process:

• Base model: Qwen/Qwen3-4B-Instruct-2507
• Modification tool: Heretic
• Heretic version: v1.4.0
• Modification method: directional ablation / abliteration
• Quantization: Q4_K_M
• Model format: GGUF

The repository also includes the relevant configuration and supporting files used for local inference.

Because model modification can depend on software versions, hardware, numerical precision, random state, and upstream implementation changes, reproducing the exact resulting weights may require recreating the original execution environment.

Run with Ollama

This repository provides a Q4_K_M GGUF distribution that can be run directly with Ollama.

Quick start

ollama run hf.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

---

Abliteration Parameters

The following parameters were obtained during the Heretic optimisation process:

These parameters describe the optimisation result used to produce this variant and should not be interpreted as general-purpose optimal parameters for other Qwen3 models.

---

Evaluation

Refusal Evaluation

An internal evaluation was performed using a 100-prompt refusal test set.

The measured KL divergence from the original model was:

Important Evaluation Disclaimer

These measurements are author-generated evaluation results and are not official Qwen benchmarks.

The refusal test set, evaluation methodology, prompt construction, sampling configuration, and evaluator implementation determine the observed results. The numbers above should therefore not be interpreted as a universal or statistically representative measure of model refusal behaviour.

In particular, the result does not mean that Abliterated will refuse exactly 16% of arbitrary prompts.

---

Upstream Qwen Performance

The following performance characteristics belong to the original Qwen3-4B-Instruct-2507 model, not to this Abliterated release.

For the official Qwen benchmark results, please refer to the upstream model card:

https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

and the Qwen3 technical report:

https://arxiv.org/abs/2505.09388

The Qwen3 technical report is the authoritative source for the Qwen3 family research and evaluation methodology.

No claim is made here that the Abliterated variant reproduces the upstream benchmark scores.

---

Technical Specifications

The upstream Qwen model is licensed under Apache License 2.0.

---

Ollama

This repository includes an Ollama-compatible configuration intended to provide correct behaviour for the non-thinking Qwen3-4B-Instruct-2507 architecture.

Direct Hugging Face usage

ollama run hf.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated:Q4_K_M

If the repository is configured with the appropriate default quantization, the model can also be invoked without explicitly specifying the quantization.

Local Modelfile

The repository includes a Modelfile for users who prefer to build the model locally.

ollama create qwen3-4b-abliterated -f Modelfile

Then:

ollama run qwen3-4b-abliterated

---

Ollama Chat Template Compatibility

During local testing, the original GGUF distribution exhibited an incompatibility with Ollama's handling of the Qwen3-4B-Instruct-2507 non-thinking architecture.

The observed behaviour included Ollama exposing:

completion
tools
thinking

and tool-call output being returned through the thinking or ordinary content channel rather than as native structured tool calls.

For example:

{
  "message": {
    "role": "assistant",
    "content": "",
    "thinking": "{\"name\":\"list_knowledge_bases\",\"arguments\":{}}\n</tool_call>"
  }
}

The Abliterated release includes a corrected Ollama serving template that removes the unintended thinking generation path and allows native tool calls to be represented through the provider's structured tool-call interface.

This is a serving/template compatibility fix. It should not be interpreted as a modification to the underlying Qwen model architecture.

---

Open WebUI

The model can be used with Open WebUI through an Ollama backend.

Recommended configuration:

Provider:
    Ollama

Model:
    qwen3-4b-abliterated

Function Calling:
    Native

Context:
    8192

Native tool calling should be preferred over legacy compatibility modes where supported.

For complex agentic workloads, users should validate tool selection, argument generation, and multi-step execution before relying on the model in production.

---

Tool Calling

Qwen3-4B-Instruct-2507 supports tool usage, and this derivative retains the underlying tool-calling capability of the base model.

The repository includes a basic tool-calling validation script:

./tests/test_ollama_tool_calling.sh qwen3-4b-abliterated

The test verifies that Ollama returns a native structured tool call rather than emitting the tool invocation as ordinary model text.

This test is intentionally small and should not be interpreted as a comprehensive evaluation of agentic reliability.

For production agentic workloads, additional evaluation should cover:

• tool selection
• argument correctness
• malformed JSON
• multi-tool execution
• tool-result interpretation
• repeated tool invocation
• prompt injection
• indirect prompt injection
• unsafe tool requests
• failure recovery

---

Recommended Inference Parameters

For general local inference, the following parameters are a reasonable starting point:

Temperature:      0.6
Top-K:            20
Top-P:            0.95
Repeat penalty:   1.0
Context:          8192

These values are intended as practical defaults for this release and are not claimed to be universally optimal.

The upstream Qwen model may recommend different sampling parameters for specific workloads. Refer to the official Qwen documentation when benchmarking the base model.

---

Context Length

The upstream Qwen3-4B-Instruct-2507 model supports a native context length of 262,144 tokens.

This release uses a default local context length of:

8192 tokens

The lower default is intentional.

Large context windows can substantially increase memory requirements, particularly when using local inference on systems with limited unified memory or GPU memory.

Users can increase the context length according to their hardware and inference runtime.

---

GGUF

The current release provides:

Qwen3-4B-Instruct-2507-Abliterated-Q4_K_M.gguf

Q4_K_M is a quantized GGUF representation intended to provide a practical balance between memory consumption, inference speed, and model quality for local deployment.

The quantized model should not be expected to reproduce the exact numerical behaviour of the original full-precision Qwen model.

---

Intended Use

Abliterated is intended primarily for:

• local experimentation
• model research
• alignment research
• abliteration research
• local LLM deployment
• agent and tool-calling experimentation
• evaluation of refusal and safety-alignment behaviour
• educational purposes
• research into model serving and inference infrastructure

The model may also be useful for security research and controlled red-team experimentation.

---

Limitations

Abliterated inherits the fundamental limitations of its base model and introduces additional uncertainty through model modification and quantization.

Potential limitations include:

• hallucinations
• incorrect factual claims
• unreliable reasoning
• malformed tool calls
• unsafe or inappropriate outputs
• inconsistent refusal behaviour
• sensitivity to prompts and system instructions
• degraded capabilities caused by abliteration
• degraded numerical fidelity caused by quantization
• unpredictable behaviour in long-context scenarios
• unreliable autonomous agent behaviour

A reduction in refusal behaviour does not imply an increase in factual accuracy, reasoning capability, or cybersecurity expertise.

The model should not be treated as an authoritative source of information.

---

Safety and Responsible Use

This model intentionally reduces aspects of the safety alignment present in the base model.

As a result, it may generate content that the original model would refuse to provide.

Users are responsible for evaluating outputs and implementing appropriate safeguards for their deployment environment.

When integrating Abliterated into an autonomous or agentic system, additional controls should be implemented around:

• tool permissions
• filesystem access
• network access
• credential handling
• secrets
• command execution
• sandboxing
• container isolation
• prompt injection
• untrusted tool output
• sensitive data handling

For security-sensitive deployments, treat model output as untrusted input.

Do not grant an LLM unrestricted access to production credentials, privileged infrastructure, sensitive data, or destructive tools solely because the model appears capable of following instructions.

---

Legal and Licensing

Base Model

Abliterated is derived from:

Qwen3-4B-Instruct-2507

Copyright and licensing for the underlying Qwen model are governed by the upstream Qwen repository and its Apache License 2.0 terms.

Official license:

https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507/blob/main/LICENSE

Apache License 2.0:

https://www.apache.org/licenses/LICENSE-2.0

Users are responsible for reviewing and complying with the applicable license terms of the upstream model and any additional software or components used with this release.

Heretic

The model modification process used:

Heretic v1.4.0

Project:

https://github.com/p-e-w/heretic

Heretic is a separate open-source project. Its source code, license, and attribution requirements are governed by its own repository and licensing terms.

This repository does not claim ownership of Heretic or of the Qwen model.

Important Notice

This model is provided as an experimental research artifact.

No additional warranty is provided by this repository beyond what may be required by applicable law or the applicable licenses of the underlying components.

Users should independently verify licensing, export-control, regulatory, organisational, and other legal requirements applicable to their intended use.

---

Credits and Attribution

This project would not exist without the work of the following projects and research:

Qwen Team

Base model:

Qwen3-4B-Instruct-2507

https://huggingface.co/Qwen/Qwen3-4B-Instruct-2507

Qwen3 technical report:

https://arxiv.org/abs/2505.09388

Heretic

Model modification:

Heretic

https://github.com/p-e-w/heretic

Heretic implements automated directional ablation and parameter optimisation for reducing safety alignment in transformer language models.

GGUF / llama.cpp ecosystem

This release is distributed in GGUF format for local inference and benefits from the broader GGML/llama.cpp ecosystem.

---

Citation

If you use Qwen3-4B-Instruct-2507-Abliterated in research, experiments, publications, or derivative work, please cite the original Qwen3 work and this repository where appropriate.

Qwen3

@misc{qwen3technicalreport,
      title={Qwen3 Technical Report},
      author={Qwen Team},
      year={2025},
      eprint={2505.09388},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.09388}
}

Abliterated

@misc{qwen3_4b_instruct_2507_abliterated,
      title={Qwen3-4B-Instruct-2507-Abliterated},
      author={IamLucif3r},
      year={2026},
      publisher={Hugging Face},
      url={https://huggingface.co/IamLucif3r/Qwen3-4B-Instruct-2507-Abliterated}
}

When publishing research based on this model, please also cite Heretic and the original Qwen3 model.

---

Repository Contents

.
├── Qwen3-4B-Instruct-2507-Abliterated-Q4_K_M.gguf
├── README.md
├── Modelfile
├── template
├── params
├── MODEL_INFO.json
├── CHANGELOG.md
├── CITATION.cff
├── tests/
│   └── test_ollama_tool_calling.sh
└── docs/
    └── reference/

---

Version

Abliterated v1.0.0

Current release:

Base model:       Qwen3-4B-Instruct-2507
Modification:     Heretic v1.4.0
Format:           GGUF
Quantization:     Q4_K_M
Default context:  8192

---

Disclaimer

Abliterated is an experimental, community-created derivative of Qwen3-4B-Instruct-2507.

The model is provided for research, experimentation, education, and local deployment.

The maintainers of this repository are not responsible for decisions made solely on the basis of model-generated output or for consequences resulting from deployment of the model in environments that lack appropriate technical, organisational, or human safeguards.

Always review the license and terms of the upstream model and other third-party components before redistribution or commercial use.