pulse-plantain

pulse-plantain

A LoRA adapter on FLUX.2 Klein 4B for remote photoplethysmography. Given a short clip of a face, the model emits a per-pixel pulse-color field decodable to instantaneous heart rate (BPM). Companion to Image Generators are Generalist Vision Learners (Gabeur et al., 2026; arXiv:2604.20329).

Hypothesis

Remote photoplethysmography (rPPG) recovers cardiac signals from sub-perceptual color variation in skin pixels. The literature spans more than fifteen years and is dominated by hand-engineered pipelines: skin-region detection, RGB-to-chrominance projections, bandpass filtering, blind source separation (Verkruysse 2008; Poh et al. 2010), and recent end-to-end CNNs (DeepPhys, MTTS-CAN). All pipelines impose explicit signal-processing inductive biases.

The test here is whether a general image generator carries enough latent representation of skin micro-color dynamics that a parameter-efficient adapter can produce the per-pixel pulse field directly, without an explicit bandpass + ICA pipeline. The output is a 2D color visualization invertible back to a metric BPM trace via a fixed bijection (analogous to deep-plantain's metric-depth encoding).

Method

Input: a short RGB face clip (5–10 seconds at standard frame rate). Output: a per-pixel pulse-color visualization frame in which color encodes the instantaneous BVP signal phase and amplitude over the local skin region. Decoded by inverting the bijection to recover a per-region BVP trace, from which BPM is estimated by peak-counting or FFT.

Training data is drawn from public rPPG datasets with synchronized ground-truth BVP (UBFC-rPPG, PURE, COHFACE, MMPD). Each training example pairs (a) a windowed face clip with (b) the corresponding ground-truth BVP trace rendered as the bijection-encoded RGB target. Evaluation reports BPM mean absolute error against held-out subjects.

Status

Placeholder. Weights and training data forthcoming.

License

Apache 2.0 — matches base FLUX.2 Klein 4B.

References

• Gabeur, V., Long, S., Peng, S., et al. Image Generators are Generalist Vision Learners. arXiv:2604.20329 (2026).
• Verkruysse, W., Svaasand, L. O., Nelson, J. S. Remote plethysmographic imaging using ambient light. Optics Express (2008).
• Poh, M.-Z., McDuff, D. J., Picard, R. W. Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Optics Express (2010).
• Liu, X., Fromm, J., Patel, S., McDuff, D. MTTS-CAN: Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement. NeurIPS (2020).
• Black Forest Labs. FLUX.2 Klein. https://bfl.ai/models/flux-2-klein (2025).