Qwen2.5-VL-3B-FakeBench-LoRA

Qwen2.5-VL-3B FakeBench LoRA

This repository contains a LoRA adapter for Qwen2.5-VL-3B-Instruct fine-tuned for binary AI-generated image detection.

The adapter predicts one of two labels:

• REAL
• AI-GENERATED

The adapter was developed as part of the project Explainable AI-Generated Image Detection with Vision-Language Models: Reasoning Strategies and Parameter-Efficient Fine-tuning.

Base Model

• Base model: Qwen/Qwen2.5-VL-3B-Instruct
• Adaptation method: LoRA
• Task: binary AI-generated image detection
• Training precision: BF16

Data Split

The fine-tuning experiment used a FakeBench-derived split:

• Training: 4,500 images
• Validation: 1,000 images
• Held-out test: 500 images
  • 250 real
  • 250 AI-generated

The fixed 500-image test set was excluded from both training and validation.

The original dataset is not distributed with this model repository. Users should consult the original FakeBench dataset source and its terms before using or redistributing dataset content.

LoRA Configuration

• Epochs: 3
• Best checkpoint: Epoch 2
• Batch size: 2
• Gradient accumulation steps: 8
• Effective batch size: 16
• Learning rate: 1e-4
• LoRA rank: 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• Target modules:
  • q_proj
  • k_proj
  • v_proj
  • o_proj
  • gate_proj
  • up_proj
  • down_proj

Training was performed on a single NVIDIA RTX PRO 6000 Blackwell 96GB GPU.

Training History

Epoch 2 achieved the lowest validation loss and was saved as the best_adapter.

Held-out Test Results

For the controlled before/after evaluation, the base model and LoRA-adapted model used the same classification-only prompt on the same held-out 500-image test set.

Under this controlled in-domain setting, LoRA fine-tuning improved:

• Accuracy by 22.0 percentage points
• Recall by 42.4 percentage points
• F1 by approximately 27.86 percentage points
• MCC from 0.6014 to 0.9880
• False negatives from 108 to 2

Important Evaluation Note

The 99.4% accuracy is an in-domain result on a held-out FakeBench split.

It should not be interpreted as evidence of broad robustness to arbitrary real-world or newly generated images. Cross-dataset and unseen-generator generalization have not yet been established.

The earlier 86.8% Qwen2.5-VL-3B result in the multi-model benchmark used a different prompt format that also requested an explanation. It is therefore not directly comparable with the 99.4% LoRA result. The controlled comparison is 77.4% Base vs 99.4% LoRA, using the same classification-only prompt.

Usage

Install the required packages:

pip install torch transformers peft qwen-vl-utils

Load the base model and this adapter:

import torch
from transformers import AutoProcessor, Qwen2_5_VLForConditionalGeneration
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-VL-3B-Instruct"
adapter_id = "NTX88666/Qwen2.5-VL-3B-FakeBench-LoRA"

processor = AutoProcessor.from_pretrained(base_model_id)

base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
)

model = PeftModel.from_pretrained(
    base_model,
    adapter_id,
)

model.eval()

The classification instruction used during fine-tuning was:

Classify this image as REAL or AI-GENERATED.
Return only one label:
REAL
AI-GENERATED

Intended Use

This adapter is intended for:

• research on AI-generated image detection;
• controlled benchmarking;
• experimentation with parameter-efficient multimodal fine-tuning;
• academic prototypes and downstream agent/tool integration.

It is not intended to be used as the sole source of truth for high-stakes authenticity, legal, moderation, or forensic decisions.

Limitations

Current limitations include:

• evaluation is currently limited to an in-domain held-out FakeBench split;
• cross-dataset generalization has not yet been evaluated;
• performance on unseen image generators has not yet been established;
• the current adapter is optimized for binary classification rather than evidence-grounded explanation generation;
• near-perfect in-domain accuracy may reflect dataset-specific visual patterns and should be interpreted cautiously;
• confidence calibration has not yet been evaluated.

Future work will focus on external generalization, unseen-generator evaluation, explanation faithfulness, and grounded visual evidence.

Project Code

Full training code, evaluation scripts, benchmark results, and experiment documentation are available on GitHub:

GitHub: https://github.com/tengxiaonong-hue/fake-image-detection-vlm

Model Repository

Hugging Face: https://huggingface.co/NTX88666/Qwen2.5-VL-3B-FakeBench-LoRA