Floppa-12B-Gemma3-Uncensored-GPTQ

Floppa-12B-Gemma3 (Uncensored) - 4-bit Quantized

Floppa-12B is a fine-tuned multimodal model based on Google's Gemma 3 12B Instruct.

This version has been quantized to 4-bit (W4A16) using llmcompressor to reduce VRAM usage and increase inference speed while maintaining high-fidelity vision capabilities.

Model Description

• Base Model: google/gemma-3-12b-it
• Architecture: 12B Dense Transformer with SigLIP Vision Encoder (Hybrid Attention).
• Quantization: W4A16 (4-bit weights, 16-bit activations) via GPTQ.
• Objective: Behavior modification (Removal of refusal guardrails + Preservation of Vision).

Quantization Details

This model was quantized using the One-Shot GPTQ method via llmcompressor.

To ensure the highest possible visual fidelity, the Vision Tower and Multimodal Projectors were excluded from quantization and remain in their original precision.

The use of Flickr30k for calibration ensures the model retains its ability to interpret complex visual scenes, unlike text-only calibration methods.

Capabilities

1. "Spicy" Vision: Capable of describing NSFW, violent, or explicit anime illustrations without triggering safety refusals.
2. Uncensored Translation: Translates dialogue exactly as written. It will use slurs, profanity, and aggressive language if the source text demands it.
3. Multimodal Context: Can translate text inside an image or describe a scene to help with translation context.

Usage (vLLM)

This model is optimized for vLLM.

Note: Ensure you are using a recent version of vLLM that supports compressed-tensors or Gemma 3 quantization.

from vllm import LLM, SamplingParams
from PIL import Image

# Load Model
# The quantization config is embedded in the model config; 
# vLLM should auto-detect the compressed-tensors format.
llm = LLM(
    model="Ryex/Floppa-12B-Gemma3-Uncensored-GPTQ",
    trust_remote_code=True,
    max_model_len=2048, # Adjust based on your VRAM
)

# Prepare Input
image = Image.open("test_image.jpg").convert("RGB")
prompt = "<image>\nDescribe this image in detail."

# Generate
inputs = {"prompt": prompt, "multi_modal_data": {"image": image}}
params = SamplingParams(temperature=1.0, max_tokens=512)

outputs = llm.generate([inputs], sampling_params=params)
print(outputs[0].outputs[0].text)

Training Data (The "Floppa Mix")

The model was fine-tuned on a balanced dataset (~10.5k rows) designed to break refusals while maintaining intelligence:

• 20% Toxic/Uncensored Text: Custom dataset of explicit dialogue and "harmful" instruction following.
• 20% Translation Skill: Unbabel/TowerBlocks-v0.2 (High-quality multilingual pairs).
• 40% General Reasoning: mlabonne/FineTome-100k (Logic and conversation).
• 20% Vision Anchors: merve/vqav2-small + Custom "Spicy" Anime Dataset SmilingWolf/camie-tagger-vs-wd-tagger-val to prevent visual catastrophic forgetting.

License & Safety

This model is built upon Gemma 3 technology from Google. Use of this model is subject to the Gemma Terms of Use. Disclaimer: This model produces uncensored content. It may generate output that is offensive, explicit, or factually incorrect. User discretion is advised. This model is intended for research, translation assistance, and creative writing workflows where content filtering is undesirable.