flux2-klein-9b-snofs-v1-4-merged

FLUX.2 Klein 9B + SNOFS v1.4 (merged)

black-forest-labs/FLUX.2-klein-9B (the distilled 9B Klein, is_distilled: true) with the SNOFS v1.4 LoKr checkpoint merged into the transformer weights at strength 1.0.

LoRA source: Ashen3/SNOFS -> Klein9b/klein_snofs_v1_4.safetensors.

Why it is merged rather than loaded as an adapter

SNOFS v1.4 ships as a direct LoKr checkpoint: 112 modules of lokr_w1 / lokr_w2 (+ alpha). diffusers has no LoKr support, and the delta kron(w1, w2) is full-rank - not the low-rank A/B decomposition the PEFT adapter machinery expects - so there is no load_lora_weights() path for it. The only way to apply it inside diffusers is an in-place merge into the transformer weights, which is what this repo contains.

Practical consequence: the LoRA is baked in at a fixed strength and cannot be re-weighted per request. Ordinary LoRA adapters still attach on top of this model normally.

How it was produced

Merged on CPU, in bfloat16, before moving to the GPU:

delta = torch.kron(w1.float(), w2.float()).reshape(target_shape)
module.weight.data.add_(delta.to(module.weight.dtype), alpha=1.0)

Fused QKV deltas are split by output dim across to_q / to_k / to_v (and add_q_proj / add_k_proj / add_v_proj), mirroring ComfyUI's direct LoKr path. alpha is intentionally ignored, as ComfyUI does on that path.

Verification of this build: 144 target modules received a delta, 0 errored, no unmatched keys, no non-finite weights. The nine modules the checkpoint does not touch - x_embedder, context_embedder, both timestep_embedder layers, proj_out, norm_out.linear and the three *_modulation.linear - are absent from the LoKr file, which only covers the attention and MLP linears inside the blocks.

Usage

import torch
from diffusers import Flux2KleinPipeline

pipe = Flux2KleinPipeline.from_pretrained(
    "sintecs/flux2-klein-9b-snofs-v1-4-merged", torch_dtype=torch.bfloat16
).to("cuda")

image = pipe(
    prompt="...",
    num_inference_steps=8,
    guidance_scale=1.0,
    generator=torch.Generator(device="cpu").manual_seed(0),
).images[0]

Needs diffusers from main - Flux2KleinPipeline is not in a tagged release yet. The base is distilled, so guidance_scale is ignored by the pipeline; 8 steps is the intended budget.

License

Use is governed by the licence of the base model (FLUX.2 Klein) and by the terms of the SNOFS LoRA. This repository adds no separate grant.