Downloads · 30 days
0
AbrahamPJ/dreamshaper-8-inpainting-qnn
dreamshaper-8-inpainting-qnn is a image-to-image model from AbrahamPJ. Use it when you need one image transformed into another. It is set up for diffusers. The card lists the license as creativeml-openrail-m.
A true 9-channel SD1.5 inpainting UNet, quantized to W8A16 and compiled to a Qualcomm QNN/HTP context binary so it runs on a phone's NPU.
Downloads · 30 days
0
Access
Public
Updated Aug 9, 2026
Repo size
685 MB
Likes
0
Public
Click a slice to open those files.
.zip685 MB · 100%
From the Hugging Face model README
A true 9-channel SD1.5 inpainting UNet, quantized to W8A16 and compiled to a Qualcomm QNN/HTP context binary so it runs on a phone's NPU.
512×512, 20 steps, ~3.9 s on a Galaxy S25 Ultra (SM8750) — about the same cost as plain txt2img on the same device.
Most on-device Stable Diffusion apps implement "inpainting" as latent blending: run an ordinary txt2img model, and after every denoising step overwrite everything outside the mask with the (re-noised) original. The UNet is never told a mask exists. That works for small texture edits but fails at the thing people actually want — it cannot remove an object, because there is no way to express "continue the background here", and during the early steps (when composition is decided) the surrounding context is itself almost pure noise, so the fill is effectively a txt2img sample clipped to the mask silhouette.
A real inpainting checkpoint takes 9 input channels instead of 4:
[ noisy latent 4 | mask 1 | masked-image latent 4 ]
so the model sees the hole and its surroundings as part of its input. This repo is that model, compiled for the NPU.
| File | Size | Notes |
|---|---|---|
unet.bin | 841 MB (881,691,224 B) | QNN context binary, dsp_arch v73 ("8gen2") |
Distributed as DreamShaper8Inpaint_qnn2.49_8gen2.zip — 654 MB
(684,721,188 B), sha256:3750b9414e428030b03de91993022f8fe512a110d3ddbee2ce1e34b422e6e15a.
This is the UNet only. It deliberately ships nothing else, because the inpaint checkpoint's text encoder and VAE are bit-identical to base DreamShaper 8 — verified tensor-by-tensor:
| Component | Tensors compared | Bit-identical | Max abs diff |
|---|---|---|---|
text_encoder | 196 | 196 | 0.0 |
vae | 248 | 248 | 0.0 |
So reuse clip_v2.mnn, pos_emb.bin, token_emb.bin, tokenizer.json,
vae_encoder.bin and vae_decoder.bin from any DreamShaper V8 QNN build (e.g.
xororz/sd-qnn →
DreamShaperV8_qnn2.28_8gen2.zip). Downloading them again would be wasted
bandwidth.
dsp_arch v73, which a
V79 device runs forward-compatibly. There is no CPU fallback.--type sd15npu_inpaint pipeline to assemble the 9-channel input
(that work is not upstream).Model directory layout:
models/inpaint/
tokenizer.json # ─┐
clip_v2.mnn # │ from any DreamShaper V8 QNN build
pos_emb.bin # │ (bit-identical, see above)
token_emb.bin # │
vae_encoder.bin # │
vae_decoder.bin # ─┘
unet.bin # ← this repo
The host must build the UNet input as concat([latent, mask, masked_latent])
in NCHW, where:
mask is the painted region downsampled to the 64×64 latent grid, 1 =
repaintmasked_latent is VAE(image * (mask < 0.5)), scaled by 0.18215[-1, 1] space (mid-grey)denoise_strength should normally be 1.0 — a partial value leaves some of
the original content in the masked region, which is occasionally useful but is
not the default people expect from inpainting.
Lykon/dreamshaper-8-inpainting → ONNX with xororz's
npuconvert attention rewrites. The only change versus the txt2img exporter
is the dummy input: randn(1, 4, 64, 64) → randn(1, 9, 64, 64). The UNet
module reads in_channels from the checkpoint config, so conv_in comes out
9-wide on its own.StableDiffusionInpaintPipeline run by hooking unet.forward, rather than
reimplementing the loop. That guarantees the recorded tensor is exactly what
the UNet is called with, including diffusers' concat order and scaling. Two
phases: full-canvas masks, then brush-like partial masks (including
background-continuation prompts, since object removal is the case that most
needs to be in-distribution). Subsampled to 400 rows.qnn-onnx-converter (W8A16,
--use_per_channel_quantization --bias_bitwidth 32 --act_bitwidth 16) →
qnn-model-lib-generator → qnn-context-binary-generator, QAIRT 2.49,
vtcm_mb 8, O 3.One calibration note worth repeating: the masked-image latents span roughly
±6.9, wider than the noisy latents (±5.6). The upstream txt2img pipeline
hardcodes a max(abs) > 7.2 outlier reject that was tuned for 4-channel input;
applied here it would discard a large fraction of the high-noise samples. This
build calibrated over the full observed range instead.
Lykon/dreamshaper-8-inpainting,
CreativeML OpenRAIL-M. That licence carries over to this conversion,
including its use restrictions.npuconvert, from
Local Dream (CC BY-NC 4.0 —
non-commercial). This repo is a converted model, but if you use the Local
Dream engine to run it, that term applies to the engine.