bdd100k-multitask-resnet18

BDD100K Multi-task ResNet-18 Model

Model Description

This is a multi-task ResNet-18 model trained on the BDD100K dataset to classify three road scene attributes simultaneously:

• Weather: 6 classes (clear, overcast, partly cloudy, rainy, snowy, foggy)
• Time of day: 3 classes (daytime, dawn/dusk, night)
• Scene type: 6 classes (highway, city street, residential, parking lot, tunnel, gas stations)

The model uses a shared ResNet-18 backbone with separate classification heads for each task, enabling efficient multi-task learning.

Model Performance

Based on validation results (30 epochs of training):

Usage

Installation

pip install torch torchvision transformers pillow

Quick Start

from transformers import AutoModel, AutoConfig
from PIL import Image
import torch

# Load the model
model = AutoModel.from_pretrained("your-username/bdd100k-multitask-resnet18", trust_remote_code=True)

# Load and predict on an image
image = Image.open("path/to/your/image.jpg")
predictions = model.predict(image)

print(f"Weather: {predictions['weather']['label']} (confidence: {predictions['weather']['confidence']:.3f})")
print(f"Time of day: {predictions['timeofday']['label']} (confidence: {predictions['timeofday']['confidence']:.3f})")
print(f"Scene: {predictions['scene']['label']} (confidence: {predictions['scene']['confidence']:.3f})")

Batch Prediction

from PIL import Image

# Load multiple images
images = [Image.open(f"image_{i}.jpg") for i in range(3)]

# Predict on batch
predictions = model.predict(images)

for i, pred in enumerate(predictions):
    print(f"Image {i+1}:")
    print(f"  Weather: {pred['weather']['label']}")
    print(f"  Time of day: {pred['timeofday']['label']}")
    print(f"  Scene: {pred['scene']['label']}")

Direct Model Usage

import torch
from torchvision import transforms

# Prepare image
transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
])

image_tensor = transform(image).unsqueeze(0)

# Forward pass
model.eval()
with torch.no_grad():
    outputs = model(image_tensor)
    
    weather_probs = torch.softmax(outputs['weather_logits'], dim=1)
    timeofday_probs = torch.softmax(outputs['timeofday_logits'], dim=1)
    scene_probs = torch.softmax(outputs['scene_logits'], dim=1)

Training Details

• Architecture: ResNet-18 with ImageNet pre-trained weights
• Dataset: BDD100K (2,722 valid samples after filtering)
• Training split: 80% training, 20% validation
• Training time: 170.18 seconds (2.84 minutes)
• Optimizer: Adam with learning rate 0.001
• Data augmentation: Random horizontal flip, rotation, color jitter
• Input size: 224x224 RGB images

Classes

Weather Classes

1. clear
2. overcast
3. partly cloudy
4. rainy
5. snowy
6. foggy

Time of Day Classes

1. daytime
2. dawn/dusk
3. night

Scene Classes

1. highway
2. city street
3. residential
4. parking lot
5. tunnel
6. gas stations

Limitations

• Model was trained on BDD100K dataset which primarily contains US driving scenes
• Performance may vary on images from different geographical regions or driving conditions
• Scene classification shows the lowest accuracy (63.49%) and may need additional training
• Model expects RGB images in driving/road scene contexts

Ethical Considerations

• This model is intended for research and educational purposes
• Should not be used as the sole decision-making tool in autonomous driving systems
• Bias may exist toward conditions represented in the BDD100K training data

Citation

If you use this model, please cite the BDD100K dataset:

@inproceedings{yu2020bdd100k,
  title={BDD100K: A diverse driving dataset for heterogeneous multitask learning},
  author={Yu, Fisher and Chen, Haofeng and Wang, Xin and Xian, Wenqi and Chen, Yingying and Liu, Fangchen and Madhavan, Vashisht and Darrell, Trevor},
  booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition},
  pages={2636--2645},
  year={2020}
}

License

This model is released under the same license as the BDD100K dataset. Please refer to the BDD100K official website for license details.

Model Card Contact

For questions about this model, please open an issue in the repository or contact the model author.