vwcar-mk8-lora

Model Card for VW Golf Mk8 - SDXL LoRA

This modelcard documents a LoRA fine-tuned Stable Diffusion XL (SDXL) model that enables personalized image generation of the Volkswagen Golf Mk8 using DreamBooth-style learning. The LoRA adapter enables high-fidelity scene synthesis of the subject car in various environmental conditions.

This modelcard is based on the official Hugging Face template.

<Gallery />

Model Details

Model Description

• Developed by: Atharva Dharmadhikari
• Model type: Text-to-image diffusion with LoRA fine-tuning
• License: CreativeML Open RAIL-M
• Finetuned from model: stabilityai/stable-diffusion-xl-base-1.0

Model Sources

• Notebook Dreambooth VW Golf
• Paper: DreamBooth ECCV 2022

Uses

Direct Use

Used to generate photorealistic images of the VW Golf Mk8 in new and diverse scenes using prompts like:

• "a mk8car driving through snow at night"
• "a mk8car on a foggy highway"
• "a red mk8car parked under city street lights"

Downstream Use

• Training AV perception models on synthetic data
• Scene simulation for CARLA or Unreal-based simulators
• Domain randomization for robustness testing

Out-of-Scope Use

• Medical imaging
• Human identity generation
• Biased prompt injection or misuse

---

Bias, Risks, and Limitations

This model reflects the biases present in the original training data of SDXL. It is also limited to generating only one vehicle identity (mk8car). It may not generalize to unseen or abstract prompts.

Recommendations

Avoid using the model in high-risk decision-making systems. Always review generated content for appropriateness and accuracy.

---

How to Get Started with the Model

from diffusers import StableDiffusionXLPipeline
import torch

pipe = StableDiffusionXLPipeline.from_pretrained(
    "stabilityai/stable-diffusion-xl-base-1.0",
    torch_dtype=torch.float16
).to("cuda")

pipe.load_lora_weights("your-username/vwcar-mk8-lora")

image = pipe("a mk8car drifting through fog", height=1024, width=1024).images[0]
image.save("mk8car_fog.png")

---

Training Details

Training Data

102 high-resolution images (1024x1024) of the Volkswagen Golf Mk8, manually collected and processed from public sources.

Training Procedure

• Mixed precision: fp16
• Optimizer: AdamW (8-bit)
• LoRA rank: 8, dropout: 0.1
• Number of steps: 1200
• Batch size: 1

Preprocessing

All images padded and resized to 1024×1024 using PIL.ImageOps.pad().

---

Evaluation

Testing Data, Factors & Metrics

Evaluated on:

• Prompt accuracy (visual + semantic)
• Visual fidelity (sharpness, composition)
• Identity preservation (same car features)

---

Environmental Impact

• Hardware Type: NVIDIA L4 (Colab)
• Hours used: ~1 hour
• Cloud Provider: Google Colab
• Compute Region: Unknown
• Carbon Emitted: Estimated < 0.1 kg CO2eq (via mlco2 calculator)

---

Technical Specifications

Model Architecture and Objective

Stable Diffusion XL with text-conditioning and UNet-based latent denoising. LoRA applied to UNet and text encoder attention layers.

Compute Infrastructure

Hardware

• GPU: NVIDIA L4 24 GB
• RAM: 16 GB (Colab VM)

Software

• PyTorch 2.1.2
• diffusers 0.25+
• transformers 4.38+
• peft, accelerate, bitsandbytes

---

Citation

BibTeX:

@article{ruiz2022dreambooth,
  title={DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation},
  author={Ruiz, Nataniel and Li, Yuanzhen and others},
  journal={ECCV},
  year={2022},
  url={https://arxiv.org/abs/2208.12242}
}

---

Model Card Authors

• [Atharva Dharmadhikari]
• Contact: [[email protected]] [atharva98]