LTX-Best-Face-ID

LTX-Best-Face-ID — LTX-2.3 Identity LoRA (Reference-to-Video / IPT2V)

An identity-preserving reference-to-video LoRA for LTX-2.3 (22B). Give it a reference photo of a person + a text prompt, and it generates a video that keeps that person's identity.

Built with overlap reference conditioning + TASS-RoPE (source-phase RoPE) and a differentiable ArcFace identity loss. Runs in ComfyUI via the companion BFS Nodes.

Status: this is the first (overlap/TASS-RoPE) model. Other experimental variants (native Gemma-vision conditioning, timestep-split texture injection) may be released later if they prove out.

---

🎬 Examples

<!-- Drop result videos/gifs in an `assets/` folder in this repo and reference them below. On the HF model page, .mp4 in a Markdown <video> tag renders inline; .gif works too. Suggested layout: reference image (left) → generated video (right), with the prompt underneath. -->

<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_1.mp4" controls autoplay loop muted></video> <video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_2.mp4" controls autoplay loop muted></video> <video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_3.mp4" controls autoplay loop muted></video>

With 2 Pass (Upscaled)

<video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_4.mp4" controls autoplay loop muted></video> <video src="https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/resolve/main/examples/sample_5.mp4" controls autoplay loop muted></video>

What it does

• Reference-to-video (ref_t2v): one reference image → video of that identity performing the prompt's action.
• Identity is injected by placing the reference latent in the target's frame-0 RoPE grid (overlap) and tagging it with a distinct source phase so the model knows it is a reference, not the first frame to generate.
• An auxiliary ArcFace face-similarity loss on the decoded prediction sharpens the identity.

---

How it works (technique)

Overlap reference + TASS-RoPE (source-phase)

The reference latent is concatenated to the video sequence sharing the frame-0 grid (classic IC-LoRA "overlap"). To stop the reference from leaking into / being confused with the generated first frame, each source gets a distinct multiplicative RoPE phase:

phase[d] = source_id · phase_scale · θ^(−d/L)      (θ = 10000)
target tokens: source_id = 0  (phase 0, exact no-op)
reference:     source_id = 2  (distinct rotary "tag")

This "source tag" lets the model separate who is who in the sequence and strongly improves identity transfer. Because the tag is positional, the same mechanism generalizes to multiple references (source_id = 2, 3, 4, …) for multi-subject conditioning.

TASS-RoPE (Temporal-Adjacent Spatial-Shifted RoPE) is from ST-DRC, Chen et al., Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation — arXiv:2606.02441. This LoRA uses an overlap + per-source RoPE-phase variant of that idea.

ArcFace identity loss

During training the flow-matching prediction is decoded to pixels, the face is aligned and passed through ArcFace (buffalo_l w600k_r50), and a cosine identity loss pulls the generated face toward the reference embedding — the same recognition space used to evaluate identity.

---

Training details

---

Reference image — what works best

This model was trained on face-focused reference images. Looking at the training data, every reference follows the same pattern:

• Close-up / bust crop — roughly chest-up, with the face large and clearly visible.
• A single subject, centered in the frame.
• Frontal or near-frontal (slight 3/4 angles are fine, but the face is always well exposed).
• Tightly cropped on the face/upper body — the training crops are about 460×406 (nearly square).

Recommendation: for the best identity transfer, give the model a reference that looks like this — crop your image to the face / upper body. Full-body shots, wide shots, or images where the face is small and far away work noticeably worse, because the model never saw references like that during training.

In short: a clean, frontal, well-lit close-up of the face is the ideal reference.

Captions (IMPORTANT)

This model was trained on captions in the ref_t2v: format that describe the scene and the person's action (not just "a person"). Identity in LTX is strongly prompt-driven — the more the prompt describes, the better the identity holds. For best results:

• Prefix prompts with ref_t2v:.
• Describe the action, setting, framing, camera in present-progressive.
• Describing identity attributes of the reference person (skin tone, hair, eyes, facial hair, glasses, face shape) noticeably improves the result.

The shared ComfyUI workflow includes a Prompt Enhancer that looks at the reference image (via the model's own multimodal Gemma-3 text encoder) and automatically enriches your prompt with the person's identity attributes — so you don't have to describe them by hand.

Example (enhanced):

ref_t2v: A light-skinned man with long dark-brown hair past his shoulders, narrow rectangular
metal-frame glasses, light blue-gray eyes and light stubble is folding clothes in a laundry room,
medium-wide shot. He places the folded clothes into a basket, then carries it across the room.
Neutral indoor lighting, simple domestic environment.

---

Usage (ComfyUI)

Requires the BFS Nodes: https://github.com/alisson-anjos/ComfyUI-BFSNodes

1. Install ComfyUI-BFSNodes (ComfyUI Manager → Install via Git URL, or clone into custom_nodes/). Dependencies (insightface, transformers, …) install automatically.
2. Load LTX-2 (checkpoint + the Gemma-3 text encoder / CLIP) as usual.
3. Add the LTX Identity Transfer (overlap + source-phase) node; feed it the reference image.
4. Load this LoRA on the MODEL path.
5. Use the shared workflow (it wires the Prompt Enhancer → identity node → sampler).

A ready-made workflow JSON is included alongside the model.

About the ArcFace projector (optional — you don't need it)

The identity node has an optional ArcFace projector dropdown (it projects a face embedding into extra conditioning tokens). You can leave it on None. In our experiments the projector's effect was marginal — identity is carried almost entirely by the overlap reference latent (and the prompt), so the projector adds very little on top and is not required for this model. It is kept only as an optional extra; skipping it keeps the graph simpler with no meaningful quality loss.

---

Tips for best results

• Use a clean, frontal, well-lit close-up as the reference — identity quality tracks reference quality closely.
• Let the Prompt Enhancer add the identity attributes (or write them yourself).
• Keep the reference framing simple; busy/occluded reference faces degrade identity.

---

Limitations

• First-frame / "mask" tendency: because identity comes from the raw reference latent, the reference appearance (and framing / accessories like glasses) can partially copy into the video — it can look like the reference is "pasted"/masked rather than freshly generated. Using a clean frontal reference and the trim-first-frame node mitigates this.
• Prompt dependence: identity is much stronger when the prompt describes the person. A bare, generic prompt gives weaker identity — use the Prompt Enhancer.
• Close-up bias: the training data skews to close-up / frontal faces, so full-body or large-angle shots hold identity less well.
• Metric caveat: ArcFace similarity is unreliable on small / turned / occluded faces — judge visually, not only by score.

---

Citation

If you use this model, please cite the base model and the technique:

@misc{ltx_best_face_id,
  title  = {LTX-Best-Face-ID: LTX-2.3 Identity LoRA (Reference-to-Video)},
  author = {Alissonerdx},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/Alissonerdx/LTX-Best-Face-ID}}
}

• Base model: LTX-2.3 (Lightricks).
• TASS-RoPE / reference conditioning — ST-DRC:

@article{chen2026stdrc,
  title  = {Spatial-Temporal Decoupled Reference Conditioning for Identity-Preserving Text-to-Video Generation},
  author = {Chen, Yuheng and Hu, Teng and Wang, Yuji and He, Qingdong and Ma, Lizhuang and Zhang, Jiangning},
  journal = {arXiv preprint arXiv:2606.02441},
  year   = {2026}
}

• Identity loss: ArcFace (Deng et al., 2019) via InsightFace buffalo_l.
• Tooling: ComfyUI-BFSNodes.

---

Acknowledgements

Trained on top of LTX-2. Identity supervision via InsightFace ArcFace. ComfyUI integration via the BFS Nodes.