StreetView360X

<video controls autoplay src="https://cdn-uploads.huggingface.co/production/uploads/661d6b110d0c77a8417334f5/anxGdT5kDoq_G9PKs1wBJ.mp4"></video>

StreetView360X is a latent diffusion model fine tuned on Stable Diffusion for generating 360 degree equirectangular street views with a single location-based prompt.

Paper
360 degree results gallery
Dataset
360 degree viewer for viewing panoramas
Short demo video
Talk video
Slides

Abstract: In this paper we introduce StreetView360X, a diffusion model for generating photorealistic equirectangular 360 degree street views from a location-based prompt. We fine tune Stable Diffusion 2.1 using a dataset of randomly sampled Google StreetView images and demonstrate superior qualitative and quantitative performance compared to vanilla stable diffusion and other diffusion-based panorama generation models. We demonstrate that diffusion based image generation models are able to learn both structural properties of images (i.e. equirectangularity) and extract geolocation-based properties from just a location name. We also show that our model is capable of generating convincing out-of-distribution street view images by combining the rich location knowledge of base Stable Diffusion and the structural knowledge of our dataset. Our results show promising potential for the future of using diffusion based models to generate content for mediums like VR, which will likely require the use of AI to fulfill the large demand for visual data. Finally, we present StreetView360AtoZ, a brand new dataset of over 6000 panoramic 360 degree street view images which can be easily extended using our scripts for fetching and downloading street views.