Downloads · 30 days
0
sentinel-opticalpattern-controlnet
Jamesbass/sentinel-opticalpattern-controlnet
sentinel-opticalpattern-controlnet is a image-to-image model from Jamesbass. Use it when you need one image transformed into another. It is set up for tensorrt. The card lists the license as other.
Prebuilt TensorRT engine that fuses the SDXL-Turbo UNet, the OpticalPattern ControlNet (Civitai 161132, v10e by nacholmo) and the SDXL IP-Adapter into the single UNet engine that Sentinel's StreamDiff node loads. It m…
Access
Public
Updated Sep 3, 2026
Repo size
8.5 GB
Likes
0
Public
Repo makeup
Click a slice to open those files.
.engine8.4 GB · 99%
At a glance
- Task
- Image-to-Image
- Library
- tensorrt
- License
- other
- Access
- Public
- Created
- Sep 3, 2026
- Updated
- Sep 3, 2026
- SHA
- 03e3c54a
Base models
Model card
From the Hugging Face model README
OpticalPattern ControlNet engine for Sentinel StreamDiff
Prebuilt TensorRT engine that fuses the SDXL-Turbo UNet, the OpticalPattern ControlNet (Civitai 161132, v10e by nacholmo) and the SDXL IP-Adapter into the single UNet engine that Sentinel's StreamDiff node loads. It makes real-time optical illusions from a live camera at 896x512.
Code, the Sentinel project, the OP_Pattern control-image Module, build scripts and tuning notes: https://github.com/jhurlbut/sentinel-opticalpattern-controlnet
Files
| File | Size | What |
|---|---|---|
profiles/sdxl/custom/opticalpattern/896x512/unet_controlnet_union_ipadapter_fp16.engine | 8.4 GB | The fused engine. Must keep this filename and folder layout inside Sentinel's engines/. |
demos/op_diffusion_cloud_face.mp4 | 58 MB | Node output: a face illusion hidden in clouds |
demos/sentinel_window_spiral_interchange.mp4 | 45 MB | Sentinel window: highway interchange bent into a spiral |
Compatibility
This is a TensorRT plan, not portable weights. It only loads on the GPU architecture it was built for:
- NVIDIA RTX 50-series (Blackwell, sm120). Built on an RTX 5090 Laptop GPU.
- TensorRT 10.15.1 runtime, which is what Sentinel 0.5.66 ships as
nvinfer_10.dll. - Sentinel 0.5.66 or newer,
engine_tier= ControlNet + IP-Adapter, FP16, 896x512. - About 9 GB of VRAM for this engine plus Sentinel's CLIP, VAE and IP-Adapter engines.
Any other GPU generation or TensorRT version: rebuild with the scripts in the GitHub repo. The build takes 30 to 60 minutes.
Install
git clone https://github.com/jhurlbut/sentinel-opticalpattern-controlnet
python sentinel-opticalpattern-controlnet/export/install_custom_pack.py --name opticalpattern --controlnet --engine <downloaded .engine> --display "SDXL OpticalPattern CN 896x512"
Then open the Sentinel project from the repo, set the StreamDiff node's engine resolution to
opticalpattern (896x512) and relaunch. Full steps and the measured tuning recipe are in the
repo README.
Binding contract
Static batch 1. Inputs sample [1,4,64,112] f16, timestep [1] f32, encoder_hidden_states
[1,81,2048] f16 (77 text + 4 IP-Adapter tokens), text_embeds [1,1280] f16, time_ids
[1,6] f32, controlnet_cond [1,3,512,896] f16, controlnet_scale [1] f32,
ipadapter_scale [70] f32. Output out_sample [1,4,64,112] f16. Verified against the
PyTorch reference at correlation 0.99993.
License
The engine is a derivative of SDXL-Turbo (Stability AI Non-Commercial Research Community License), the OpticalPattern ControlNet (CreativeML Open RAIL++-M with the author's addendum) and IP-Adapter (Apache-2.0). It is provided for non-commercial research use only; commercial use needs a Stability AI license. The ControlNet training weights themselves are not redistributed here.
- Task
- Image-to-Image
- Library
- tensorrt
- License
- other
- Created
- Sep 3, 2026
- Updated
- Sep 3, 2026