LumaFlux

LumaFlux: Lifting 8-Bit Worlds to HDR Reality with Physically-Guided Diffusion Transformers

Paper (arXiv:2604.02787)  ·  Code (GitHub)  ·  Project page  ·  ComfyUI nodes

Shreshth Saini, Hakan Gedik, Neil Birkbeck, Yilin Wang, Balu Adsumilli, Alan C. Bovik (The University of Texas at Austin and Google, Inc.)

LumaFlux converts 8-bit SDR (BT.709) images and video into 10-bit HDR (PQ, BT.2020) by adapting a frozen FLUX.1-dev MM-DiT. This repository holds the trained adapter weights only: 71,315,893 parameters, 0.57% of the backbone. The frozen FLUX.1-dev backbone and SigLIP encoder are downloaded separately at load time.

Conversion is prompt-free. Inference integrates a rectified-flow bridge from the SDR latent in 8 steps, and video is handled by the same image model with no temporal layers, optical flow, or video fine-tuning.

Checkpoints

Both files carry the identical architecture and are interchangeable at load time; they differ only in training data and schedule.

Results

Luma-Eval (our common run-by-us protocol; 100 held-out frames, synthetic tone-mapping and codec degradations). lumaflux-main:

HDRTV1K (117 published test pairs, evaluated under the authors' protocol):

The zero-shot row is reported for transparency: the main model never sees the HDRTV1K training split, so it is not comparable to methods trained on it.

Video. Sharing one bridge-noise realization across frames and smoothing the applied tone-curve parameters with an EMA reduces signed excess PU21 flicker by 53.3%, with pooled temporal energy within 3.3% of the reference, without any temporal training.

Cost. 8 solver steps, 5.28 s per 1080p frame and 27.1 GB peak memory on a single GH200. Forty steps cost 4.7x more without improving quality.

Usage

pip install "lumaflux @ git+https://github.com/shreshthsaini/LumaFlux.git"
huggingface-cli download shreshthsaini/LumaFlux lumaflux-main.safetensors --local-dir weights

# Image or video in, 10-bit PQ/BT.2020 HDR out
python -m lumaflux.inference.cli \
  --config configs/model/flux_dev.yaml \
  --adapters weights/lumaflux-main.safetensors \
  --input input_sdr.mp4 --output output_hdr.mp4 --steps 8

Loading the adapters also pulls black-forest-labs/FLUX.1-dev (gated; accept its license first) and google/siglip-so400m-patch14-384. Full instructions, data preparation, training, and the evaluation suite are in the GitHub repository.

In ComfyUI, the GitHub repository doubles as a custom node pack: clone it into custom_nodes, install its requirements.txt, and the nodes run the same pipeline, reusing the flux1-dev.safetensors and ae.safetensors files already in your model folders. These adapters download into models/lumaflux/ on first use. Details in comfyui/README.md.

Training

Both checkpoints were trained with a frozen backbone: only PGA (gated low-rank attention residuals), PCM (SigLIP FiLM conditioning), the HDR Residual Coupler, the timestep-layer conditioner, and the monotone RQS tone head receive gradients. Optimization used AdamW at learning rate 2e-4, global batch 16, 512px crops, bf16, and a rectified-flow velocity objective with decoded-reconstruction and spline-smoothness terms. The main model trained for about 36 hours on four GH200 GPUs (~144 GPU-hours).

Training data combines CHUG user-generated HDR, LIVE-TMHDR (including expert-graded SDR), HDRTV1K, and the Netflix Open Content Sol Levante HDR10 master. SDR inputs are either genuine expert grades or generated with a frozen degradation chain of eight tone-mapping operators, BT.2020-to-BT.709 gamut conversion, and three CRF settings.

License and intended use

These adapters only function on top of FLUX.1-dev and are therefore released under the FLUX.1-dev Non-Commercial License; research use only. The accompanying source code is Apache-2.0. Users must separately accept the FLUX.1-dev license.

Intended for research on inverse tone mapping, archival SDR remastering, and HDR video pipelines. Not validated for medical, forensic, or safety-critical imaging.

Limitations

• The frozen VAE was not trained as an HDR codec, so the tone head calibrates its output but cannot recover detail absent from heavily clipped SDR.
• Temporal stabilization is inference-time only and does not model long-range motion explicitly.
• On the selected qualitative scenes the outputs measure slightly more saturated than their references (mean ITP chroma 60.65 versus 49.80).
• Correct display metadata and EOTF settings are required at playback.

Citation

@article{saini2026lumaflux,
  title   = {LumaFlux: Lifting 8-Bit Worlds to HDR Reality with Physically-Guided Diffusion Transformers},
  author  = {Saini, Shreshth and Gedik, Hakan and Birkbeck, Neil and Wang, Yilin and Adsumilli, Balu and Bovik, Alan C.},
  journal = {arXiv preprint arXiv:2604.02787},
  year    = {2026}
}