pixeldit-controlnet

madtune/pixeldit-controlnet

pixeldit-controlnet is a image-to-image model from madtune. Use it when you need one image transformed into another. It is set up for diffusers. The card lists the license as apache-2.0.

ControlNet scribble conditioning and IP-Adapter style transfer for PixelDiT-1300M.

Downloads · 30 days

28% of all-time downloads

All-time downloads

191

Public

Repo size

1.8 GB

Likes

Public

Hugging Face

Repo makeup

Click a slice to open those files.

.safetensors1.8 GB · 100%

At a glance

Task: Image-to-Image
Library: diffusers
License: apache-2.0
Model type: pixeldit_controlnet
Access: Public
Created: Jun 22, 2026
Updated: Jun 22, 2026
SHA: db700ca4

Base models

madtune/pixeldit-diffusers

Spaces

madtune/pixeldit-space

Model card

From the Hugging Face model README

PixelDiT ControlNet + IP-Adapter

ControlNet scribble conditioning and IP-Adapter style transfer for PixelDiT-1300M.

Note: PixelDiT-1300M is a model by NVIDIA Research. This repo contains trained adapters only — we are not affiliated with NVIDIA.

Files

File	Description
`controlnet.safetensors`	Combined ControlNet (7 blocks) + IP-Adapter weights
`ip_adapter.safetensors`	IP-Adapter weights only
`hed_detector.safetensors`	HED edge detector (Apache-2.0, VGG-based)
`config.json`	Model config
`train.py`	Joint ControlNet + IP-Adapter training script
`precompute_wd_tags.py`	Run WD tagger on dataset → `wd_tags.json`
`precompute_embeddings.py`	Encode images with SigLIP + Gemma → memmap files
`precompute_hed.py`	Precompute HED edge maps for a dataset
`control_maps.py`	Edge map post-processing utilities
`hed.py`	HED model definition
`convert_to_safetensors.py`	Convert .pt checkpoints to safetensors

Usage

from diffusers.pipelines.pixeldit import PixelDiTStyledPipeline
from huggingface_hub import hf_hub_download
from PIL import Image
import torch

pipe = PixelDiTStyledPipeline.from_pretrained_styled(
    "madtune/pixeldit-diffusers",
    controlnet_path=hf_hub_download("madtune/pixeldit-controlnet", "controlnet.safetensors"),
    ip_adapter_path=hf_hub_download("madtune/pixeldit-controlnet", "ip_adapter.safetensors"),
    hed_ckpt_path=hf_hub_download("madtune/pixeldit-controlnet", "hed_detector.safetensors"),
    torch_dtype=torch.bfloat16,
)
pipe.enable_model_cpu_offload(gpu_id=1)

out = pipe(
    image=Image.open("style_ref.jpg"),
    prompt="gothic pale woman, dramatic rim lighting",
    variation_strength=0.85,
    ctrl_strength=0.25,
    ip_strength=0.85,
    flow_shift=8.0,
    guidance_scale=4.5,
    num_inference_steps=50,
).images[0]
out.save("output.jpg")

Recommended settings

Mode	`ctrl_strength`	`ip_strength`	`variation_strength`
Pure variation	0.0	0.0	0.65–0.85
ControlNet only	0.25	0.0	0.85
IP-Adapter only	0.0	0.85	0.85
Full combo (best)	0.25	0.35–0.85	0.85

flow_shift=8.0 + guidance_scale=3.0–3.5 works well at 768px+. 4.5 is valid but produces oversaturated colours.

Task: Image-to-Image
Library: diffusers
Type: pixeldit_controlnet
License: apache-2.0
Created: Jun 22, 2026
Updated: Jun 22, 2026