RoadScene-VLM-Qwen2.5-VL-3B-QLoRA

RoadScene-VLM QLoRA Adapter

This repository contains a QLoRA adapter for:

Qwen/Qwen2.5-VL-3B-Instruct

It was fine-tuned on a small BDD100K road-scene pilot dataset for structured multimodal road-scene understanding.

Task

Given a road image, the model outputs strict JSON containing:

• weather
• time of day
• scene type
• cars
• buses
• trucks
• pedestrians
• traffic lights
• traffic signs

Example output:

{
  "weather": "overcast",
  "time_of_day": "daytime",
  "scene": "city street",
  "cars": 12,
  "buses": 0,
  "trucks": 1,
  "pedestrians": 2,
  "traffic_lights": 3,
  "traffic_signs": 4
}

Training Configuration

• Base model: Qwen/Qwen2.5-VL-3B-Instruct
• Train / Validation / Test: 800 / 100 / 200
• Epochs: 1
• Quantization: 4-bit NF4
• Double quantization: enabled
• Compute dtype: BF16
• LoRA rank: 16
• LoRA alpha: 32
• LoRA dropout: 0.05
• Target modules: q_proj, k_proj, v_proj, o_proj
• Learning rate: 1e-4
• Effective batch size: 8
• GPU: NVIDIA A100

Frozen Test Results

Key Findings

QLoRA substantially improved road-weather label alignment, strict JSON output compliance, and vehicle-count prediction.

For vehicle counting:

• MAE decreased from 7.00 to 3.14
• Spearman correlation increased from 0.517 to 0.764
• prediction bias changed from -7.00 to +0.48

The QLoRA model also outperformed image-independent mean and median count baselines, suggesting that the improvement was not explained only by learning the dataset's average vehicle count.

Limitations

This is a small pilot experiment rather than a production autonomous-driving model.

The dataset is strongly imbalanced, and several rare classes have insufficient representation in the natural test set.

Scene classification showed increased majority-class bias toward city street.

The visual backbone was not directly fine-tuned.

Only one primary QLoRA configuration was evaluated, so the experiment does not establish an optimal hyperparameter setting.

Base Model

Qwen/Qwen2.5-VL-3B-Instruct

Please review the upstream model repository and license terms before reuse.