Downloads · 30 days
30
16% of all-time downloads
granite-3.1-8b-instruct-abliterated-exl2
cgus/granite-3.1-8b-instruct-abliterated-exl2
granite-3.1-8b-instruct-abliterated-exl2 is a text generation model from cgus. Use it when you need the model to write or continue text. It is set up for transformers. The card lists the license as apache-2.0.
Model: granite-3.1-8b-instruct-abliterated Made by: huihui-ai Granite 3 authors: Granite Team, IBM
All-time downloads
192
Public
Repo size
29 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.safetensors4.5 GB · 100%
At a glance
- Task
- Text Generation
- Library
- transformers
- License
- apache-2.0
- Model type
- granite
- Access
- Public
- Created
- Feb 3, 2025
- Updated
- Feb 3, 2025
- SHA
- 300207fd
Try a prompt
Base models
Model card
From the Hugging Face model README
granite-3.1-8b-instruct-abliterated-exl2
Model: granite-3.1-8b-instruct-abliterated
Made by: huihui-ai
Granite 3 authors: Granite Team, IBM
Quants
4bpw h6 (main)
4.5bpw h6
5bpw h6
6bpw h6
8bpw h8
Quantization notes
Made with exllamav2 0.2.7 with default dataset. This model requires exllamav2 0.2.7 or newer.
Exl2 quants require to be fully loaded into GPU VRAM, RAM offloading isn't supported natively.
Additionally it requires Nvidia RTX on Windows or Nvidia RTX/AMD ROCm on Linux.
These quants can be used with TabbyAPI or Text-Generation-WebUI.
Original model card
huihui-ai/granite-3.1-8b-instruct-abliterated
This is an uncensored version of ibm-granite/granite-3.1-8b-instruct created with abliteration (see remove-refusals-with-transformers to know more about it).
This is a crude, proof-of-concept implementation to remove refusals from an LLM model without using TransformerLens.
Use with ollama
You can use huihui_ai/granite3.1-dense-abliterated directly,
ollama run huihui_ai/granite3.1-dense-abliterated
- Task
- Text Generation
- Library
- transformers
- Type
- granite
- License
- apache-2.0
- Created
- Feb 3, 2025
- Updated
- Feb 3, 2025