Qwen2.5-VL-7B-Instruct-abliterated_EXL2_8.0bpw

huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated

This is an uncensored version of Qwen/Qwen2.5-VL-7B-Instruct created with abliteration (see remove-refusals-with-transformers to know more about it).

It was only the text part that was processed, not the image part.

Usage

You can use this model in your applications by loading it with Hugging Face's transformers library:

from transformers import Qwen2_5_VLForConditionalGeneration, AutoTokenizer, AutoProcessor
from qwen_vl_utils import process_vision_info

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    "huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated", torch_dtype="auto", device_map="auto"
)
processor = AutoProcessor.from_pretrained("huihui-ai/Qwen2.5-VL-7B-Instruct-abliterated")

image_path = "/tmp/test.png"

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": f"file://{image_path}",
            },
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

text = processor.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)
image_inputs, video_inputs = process_vision_info(messages)
inputs = processor(
    text=[text],
    images=image_inputs,
    videos=video_inputs,
    padding=True,
    return_tensors="pt",
)
inputs = inputs.to("cuda")

generated_ids = model.generate(**inputs, max_new_tokens=256)
generated_ids_trimmed = [
    out_ids[len(in_ids) :] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
output_text = output_text[0]

print(output_text)

Donation

Your donation helps us continue our further development and improvement, a cup of coffee can do it.

• bitcoin:

  bc1qqnkhuchxw0zqjh2ku3lu4hq45hc6gy84uk70ge