Reading Notes

Personal reading notes. Original paper: arXiv:2507.14885v1.

arxiv_id: 2507.14885v1

BeatFormer: Motion-Robust Remote Heart Rate Estimation Based on Unsupervised Spectral Zoomed Attention

1. Paper Basic Information

2. Research Background: Dilemmas and Opportunities of rPPG

Remote Photoplethysmography (rPPG) technology captures cardiac signals through facial video, holding significant potential in clinical monitoring and human-computer interaction. However, current rPPG research faces a core contradiction: the trade-off between the generalization of data-driven methods and the robustness of handcrafted feature methods.

• Limitations of Deep Learning: Although deep learning models have made significant progress in rPPG estimation, they rely heavily on large-scale, diverse datasets for training. This leads to reduced generalization capabilities when facing biases such as skin tone, lighting changes, video compression, and body motion, while also incurring high computational costs.
• Shortcomings of Handcrafted Methods: In contrast, traditional handcrafted feature methods (e.g., CHROM, POS) utilize physiological prior knowledge and demonstrate better generalization and computational efficiency in unseen scenarios (e.g., motion scenarios). However, they are typically based on linear assumptions and struggle to capture complex nonlinear physiological relationships, limiting their performance under complex conditions.
• Necessity of Hybrid Methods: To combine the advantages of both, the authors propose a hybrid method. It requires the powerful pulsation information extraction capability of deep learning, as well as the physiological priors of traditional methods to enhance robustness against motion.

Furthermore, existing frequency-domain methods typically rely on the Fast Fourier Transform (FFT), which faces a resolution trade-off problem, making it difficult to ensure motion separation within short time windows. Recent research has introduced the Chirp-Z Transform (CZT), allowing for adjustable resolution spectral analysis within specific frequency bands (e.g., heart rate bandwidth), providing a new idea for improving estimation accuracy under short windows.

3. Core Method: Detailed Explanation of BeatFormer

To address the above issues, the authors propose BeatFormer, a lightweight spectral attention model. Its core innovation lies in integrating Zoomed Orthonormal Complex Attention (ZOCA) and Frequency Domain Energy Measurement, along with introducing an unsupervised Spectral Contrastive Learning (SCL) strategy.

3.1 Overall Architecture

The overall flow of BeatFormer is designed to separate pulsation information from RGB video trajectories while suppressing motion interference. The model first segments and overlaps RGB trajectories, converting them to the frequency domain using CZT. Subsequently, pulsation information is filtered through the BeatFormer module, combined with orthogonal regularization and energy-based weighting. Finally, it is converted back to the time domain via Inverse CZT (ICZT), and the rPPG signal is reconstructed through an overlap-add operation.

During the training phase, Spectral Contrastive Learning (SCL) is adopted, utilizing meaningful transformations in the frequency domain to enforce explicit priors, thereby achieving label-free training.

Figure 1: BeatFormer Overall Architecture (from PDF page 3)

3.2 Key Components: CZT and Spectral Attention

Chirp-Z Transform (CZT)

Traditional Discrete Fourier Transform (DFT) samples at equal intervals on the unit circle in the z-plane, whereas CZT allows calculating the z-transform along a general spiral contour in the z-plane. This makes CZT unconstrained by the unit circle, enabling “zoomed” analysis targeting specific frequency bands (e.g., heart rate range).

Its mathematical definition is as follows: $CZT(x[n]) = \sum_{n=0}^{N-1} x[n] \cdot z_k^{-n}$ where $z_k = A \cdot W^{-k}$, $A$ is the complex starting point, and $W$ is a complex scalar describing the complex ratio between contour points. This flexibility allows CZT to provide higher frequency resolution within short time windows, thereby more accurately separating heart rate signals from motion noise.

Figure 2: CZT Transform Principle Schematic (from PDF page 5)

Zoomed Orthonormal Complex Attention (ZOCA)

The core module of BeatFormer is ZOCA. Unlike traditional spatiotemporal attention, ZOCA operates in the frequency domain. It learns spectral filters to separate pulsation information from motion changes.

• Implicit Physiological Priors: The model does not rely on unconstrained data-driven weights but combines implicit physiological priors, enhancing robustness to training data noise (e.g., lighting changes, motion).
• Computational Efficiency: Compared to existing models, ZOCA reduces the number of parameters and computational costs while maintaining powerful feature extraction capabilities.

Figure 3: ZOCA Module Details and Spectral Filtering Mechanism (from PDF page 6)

3.3 Unsupervised Training: Spectral Contrastive Learning (SCL)

To eliminate dependence on PPG or heart rate labels, the authors propose Spectral Contrastive Learning (SCL).

• Explicit Physiological Assumptions: SCL is based on explicit physiological assumptions, enhancing model training by maximizing intra-class similarity and minimizing inter-class differences.
• Frequency Domain Transformations: Unlike traditional spatiotemporal augmentation, SCL focuses on providing video transformations that yield meaningful frequency representations.
• Effectiveness: This method enables BeatFormer to achieve performance comparable to supervised methods without labels, and even demonstrates stronger robustness under significant motion distortion.

4. Experimental Verification and Performance Analysis

The authors validated the effectiveness of BeatFormer on multiple public rPPG benchmark datasets, including PURE, UBFC-rPPG, and MMPD.

• Cross-Dataset Evaluation: Experiments focused on demonstrating the model’s performance in cross-dataset evaluations, particularly in motion scenarios.
• Motion Robustness: Results indicate that BeatFormer outperforms many existing deep learning methods in handling motion interference, thanks to its spectral attention mechanism combined with physiological priors.
• Balance of Efficiency and Performance: The model achieves high-precision heart rate estimation while maintaining a lightweight profile (few parameters, fast computation).

Figure 4: Performance Comparison Results Across Different Datasets (from PDF page 8)

Figure 5: Robustness Analysis in Motion Scenarios (from PDF page 8)

5. Summary and Implications

The BeatFormer paper proposes a novel hybrid method, successfully combining the nonlinear modeling capabilities of deep learning with the physiological priors of traditional methods.

• Main Contributions:

  1. Proposed a lightweight spectral filter Transformer (BeatFormer), integrating Zoomed Orthonormal Complex Attention and frequency domain energy measurement.
  2. Introduced a frequency domain self-contrastive learning method based on explicit physiological assumptions, achieving robust training without labels.
  3. Verified the advantages of frequency domain rPPG estimation in motion scenarios through extensive experiments, particularly leveraging the properties of CZT.

• Future Outlook: This research provides a new direction for the rPPG field, namely using frequency domain transformations (such as CZT) and attention mechanisms to solve motion robustness problems. Future work can further explore the integration of more complex physiological priors and generalization capabilities in more diverse real-world scenarios (e.g., low light, occlusion).