Downloads · 30 days
180
10% of all-time downloads
logos-flux/Llama-3.1-8B-Instruct-heretic-GGUF
Llama-3.1-8B-Instruct-heretic-GGUF is a machine learning model from logos-flux. Use it for the machine learning task on the model card, and read the license before you ship it in a product. The card lists the license as llama3.1.
GGUF quantized versions of p-e-w/Llama-3.1-8B-Instruct-heretic from "The Bestiary" collection.
Downloads · 30 days
180
10% of all-time downloads
All-time downloads
1.8K
Public
Repo size
35.3 GB
Likes
0
Public
Click a slice to open those files.
.gguf35.3 GB · 100%
From the Hugging Face model README
GGUF quantized versions of p-e-w/Llama-3.1-8B-Instruct-heretic from "The Bestiary" collection.
This is a GGUF conversion of the Llama-3.1-8B-Instruct-heretic model, which is an abliterated (uncensored) version of Meta's Llama 3.1 8B Instruct model. The model has had its refusal mechanisms removed, making it more willing to engage with any prompt.
Original Model: p-e-w/Llama-3.1-8B-Instruct-heretic Collection: The Bestiary by p-e-w
This repository contains 4 quantization levels:
| File | Size | Description | Use Case |
|---|---|---|---|
llama-3.1-8b-instruct-heretic-f16.gguf | 15GB | Full 16-bit precision | Best quality, highest memory usage |
llama-3.1-8b-instruct-heretic-Q8_0.gguf | 8.0GB | 8-bit quantization | High quality, good balance |
llama-3.1-8b-instruct-heretic-Q5_K_M.gguf | 5.4GB | 5-bit quantization | Balanced quality/size |
llama-3.1-8b-instruct-heretic-Q4_K_M.gguf | 4.6GB | 4-bit quantization | Smallest size, good quality |
Recommended: Q4_K_M for most users (best balance of quality and size)
FROM ./llama-3.1-8b-instruct-heretic-Q4_K_M.gguf
TEMPLATE """{{ if .System }}<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>{{ end }}{{ if .Prompt }}<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>{{ end }}<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>"""
PARAMETER stop "<|start_header_id|>"
PARAMETER stop "<|end_header_id|>"
PARAMETER stop "<|eot_id|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 8192
ollama create llama-3.1-8b-heretic:Q4_K_M -f Modelfile
ollama run llama-3.1-8b-heretic:Q4_K_M
./llama-cli -m llama-3.1-8b-instruct-heretic-Q4_K_M.gguf -p "Your prompt here" -n 512
Once imported to Ollama, the model will automatically appear in the Open WebUI model dropdown.
This is an uncensored model with refusal mechanisms removed. Use responsibly and in accordance with applicable laws and regulations.
Inherits the Llama 3.1 Community License from the base model.