marigold-normals-v1-1

<h1 align="center">Marigold Normals v1-1 Model Card</h1> <p align="center"> <a title="Image Normals" href="https://huggingface.co/spaces/prs-eth/marigold-normals" target="_blank" rel="noopener noreferrer" style="display: inline-block;"> <img src="https://img.shields.io/badge/%F0%9F%A4%97%20Image%20Normals%20-Demo-yellow" alt="Image Normals"> </a> <a title="diffusers" href="https://huggingface.co/docs/diffusers/using-diffusers/marigold_usage" target="_blank" rel="noopener noreferrer" style="display: inline-block;"> <img src="https://img.shields.io/badge/%F0%9F%A4%97%20diffusers%20-Integration%20🧨-yellow" alt="diffusers"> </a> <a title="Github" href="https://github.com/prs-eth/marigold" target="_blank" rel="noopener noreferrer" style="display: inline-block;"> <img src="https://img.shields.io/github/stars/prs-eth/marigold?label=GitHub%20%E2%98%85&logo=github&color=C8C" alt="Github"> </a> <a title="Website" href="https://marigoldcomputervision.github.io/" target="_blank" rel="noopener noreferrer" style="display: inline-block;"> <img src="https://img.shields.io/badge/%E2%99%A5%20Project%20-Website-blue" alt="Website"> </a> <a title="arXiv" href="https://arxiv.org/abs/2505.09358" target="_blank" rel="noopener noreferrer" style="display: inline-block;"> <img src="https://img.shields.io/badge/%F0%9F%93%84%20Read%20-Paper-AF3436" alt="arXiv"> </a> <a title="Social" href="https://twitter.com/antonobukhov1" target="_blank" rel="noopener noreferrer" style="display: inline-block;"> <img src="https://img.shields.io/twitter/follow/:?label=Subscribe%20for%20updates!" alt="Social"> </a> <a title="License" href="https://huggingface.co/stabilityai/stable-diffusion-2/blob/main/LICENSE-MODEL" target="_blank" rel="noopener noreferrer" style="display: inline-block;"> <img src="https://img.shields.io/badge/License-OpenRAIL++-929292" alt="License"> </a> </p>

This is a model card for the marigold-normals-v1-1 model for monocular normals estimation from a single image. The model is fine-tuned from the stable-diffusion-2 model as described in our papers:

• CVPR'2024 paper titled "Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation"
• Journal extension titled "Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis"

Using the model

• Play with the interactive Hugging Face Spaces demo: check out how the model works with example images or upload your own.
• Use it with diffusers to compute the results with a few lines of code.
• Get to the bottom of things with our official codebase.

Model Details

• Developed by: Bingxin Ke, Kevin Qu, Tianfu Wang, Nando Metzger, Shengyu Huang, Bo Li, Anton Obukhov, Konrad Schindler.
• Model type: Generative latent diffusion-based normals estimation from a single image.
• Language: English.
• License: CreativeML Open RAIL++-M License.
• Model Description: This model can be used to generate an estimated surface normals map of an input image.
  • Resolution: Even though any resolution can be processed, the model inherits the base diffusion model's effective resolution of roughly 768 pixels. This means that for optimal predictions, any larger input image should be resized to make the longer side 768 pixels before feeding it into the model.
  • Steps and scheduler: This model was designed for usage with DDIM scheduler and between 1 and 50 denoising steps.
  • Outputs:
    • Surface normals map: The predicted values are 3-dimensional unit vectors in the screen space camera.
    • Uncertainty map: Produced only when multiple predictions are ensembled with ensemble size larger than 2.
• Resources for more information: Project Website, Paper, Code.
• Cite as:

@misc{ke2025marigold,
  title={Marigold: Affordable Adaptation of Diffusion-Based Image Generators for Image Analysis},
  author={Bingxin Ke and Kevin Qu and Tianfu Wang and Nando Metzger and Shengyu Huang and Bo Li and Anton Obukhov and Konrad Schindler},
  year={2025},
  eprint={2505.09358},
  archivePrefix={arXiv},
  primaryClass={cs.CV}
}

@InProceedings{ke2023repurposing,
  title={Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation},
  author={Bingxin Ke and Anton Obukhov and Shengyu Huang and Nando Metzger and Rodrigo Caye Daudt and Konrad Schindler},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2024}
}