Downloads · 30 days
0
0% of all-time downloads
YifeiDevs/Huihui-Qwen3-8B-abliterated-v2-FP8-Comfy
Huihui-Qwen3-8B-abliterated-v2-FP8-Comfy is a machine learning model from YifeiDevs. Use it for the machine learning task on the model card, and read the license before you ship it in a product. It is set up for transformers.
Load CLIP - clipname: Huihui-Qwen3-8B-abliterated-v2-FP8.safetensors - type: flux2 - device: default
Downloads · 30 days
0
0% of all-time downloads
All-time downloads
8
Public
Repo size
8.2 GB
Likes
7
Public
Click a slice to open those files.
.safetensors8.2 GB · 100%
From the Hugging Face model README
Load CLIP
FP8 quant of Huihui-Qwen3-8B-abliterated-v2. Safety refusals have been removed (abliterated). User assumes all legal responsibility for its use and output.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
MODEL_ID = "huihui-ai/Huihui-Qwen3-8B-abliterated-v2"
SAVE_DIR = "Huihui-Qwen3-8B-abliterated-v2-FP8-Comfy"
print("Loading model...")
# Load as BF16, convert on CPU to save VRAM
model = AutoModelForCausalLM.from_pretrained(
MODEL_ID,
torch_dtype=torch.bfloat16,
device_map="cpu",
trust_remote_code=True
)
print("Converting to FP8 (E4M3)...")
# Iterate through parameters and convert to float8_e4m3fn
for name, param in model.named_parameters():
# Only convert Linear layer weights, skip LayerNorm and Bias for precision
if "weight" in name and param.ndim == 2:
param.data = param.data.to(torch.float8_e4m3fn)
print(f"Saving to {SAVE_DIR}...")
# Save with safetensors, max 20GB per file
model.save_pretrained(
SAVE_DIR,
safe_serialization=True,
max_shard_size="10GB"
)
print("Done!")