Lift3Dreamer

Usage:

from diffusers import StableDiffusionInpaintPipeline
import torch
from diffusers.utils import load_image, make_image_grid
import PIL

# 指定模型文件路径
model_path = "Liangyingping/Lift3Dreamer"
# 加载模型
pipe = StableDiffusionInpaintPipeline.from_pretrained(
    model_path, torch_dtype=torch.float16
)
pipe.to("cuda")  # 如果有 GPU，可以将模型加载到 GPU 上

init_image = load_image("assets/debug_masked_image.png")
mask_image = load_image("assets/debug_mask.png")
W, H = init_image.size

prompt = "a photo of a person"
image = pipe(
    prompt=prompt,
    image=init_image,
    mask_image=mask_image,
    h=512, w=512
).images[0].resize((W, H))

print(image.size, init_image.size)

image2save = make_image_grid([init_image, mask_image, image], rows=1, cols=3)
image2save.save("image2save_ours.png")

If you use this inpainting model, please cite our paper:

@article{LIANG2026,
title = {Lift3Dreamer: Boosting Text-Driven Novel View Synthesis via Lifted 3D Inpainting Model from Single Images.},
journal = {Fundamental Research},
year = {2026},
issn = {2667-3258},
author = {Yingping Liang and Ying Fu and Jiaming Liu and Debing Zhang},
}