VitalLens 2.0: High-Fidelity rPPG for Heart Rate Variability Estimation from Face Video

Reading Notes

Personal reading notes. Original paper: arXiv:2510.27028v1.

arxiv_id: 2510.27028v1

Paper Deep Dive: VitalLens 2.0 - High-Fidelity rPPG Heart Rate Variability Estimation

1. Paper Basic Information

• Paper Title: VitalLens 2.0: High-Fidelity rPPG for Heart Rate Variability Estimation from Face Video
• Paper ID: 2510.27028v1
• Author: Philipp V. Rouast (Rouast Labs)
• Core Field: Computer Vision, Remote Photoplethysmography (rPPG), Physiological Signal Estimation
• Main Goal: Propose a novel deep learning model capable of reconstructing physiological waveforms from facial video with high fidelity, thereby accurately estimating Heart Rate (HR), Respiratory Rate (RR), and Heart Rate Variability (HRV) metrics.

2. Research Background and Motivation

Remote Photoplethysmography (rPPG) utilizes standard video signals to estimate physiological information, offering significant potential for non-invasive health monitoring. The authors’ previous version, VitalLens 1.0, already achieved real-time estimation of heart rate and respiratory rate. However, VitalLens 2.0 aims to bridge the gap from simple “rate estimation” (i.e., finding the dominant frequency) to “high-fidelity waveform reconstruction.”

The core challenge in this transition is that accurate analysis of Heart Rate Variability (HRV) requires sub-second temporal accuracy of Inter-Beat Intervals (IBIs). Traditional rPPG methods often struggle to capture subtle beat-to-beat variations, leading to inaccurate HRV metric estimation. VitalLens 2.0 addresses this challenge by improving the model architecture and expanding the scale of training data, striving for more robust physiological waveform reconstruction.

3. Core Contributions and Innovations

The significant improvement of VitalLens 2.0 is primarily attributed to the following two key developments:

1. Large-Scale and Carefully Curated Training Dataset:

   • Combines internal data (PROSIT) and public datasets (Vital Videos).
   • All samples are manually curated to ensure high quality of both video and labels.
   • The final training set includes 1,413 unique individuals, representing a significant increase in scale and diversity compared to VitalLens 1.0.
   • Data covers diverse shooting locations, lighting conditions, camera types, backgrounds, and unscripted participant behaviors (including significant camera motion caused by handheld devices).

2. Novel Model Architecture and Training Methods:

   • Built upon an EfficientNet backbone to create an end-to-end deep convolutional neural network.
   • Introduces novel temporal-attentive mechanisms specifically designed to extract high-fidelity physiological waveforms.
   • The design focuses on minimizing signal noise and preserving the precise temporal location of systolic peaks, which is a critical prerequisite for reliably extracting IBIs and conducting HRV analysis.

4. Dataset Details

To verify the improvements, the authors established a new, large-scale combined test set. This test set is completely participant-disjoint from the training and validation sets to ensure robust and unbiased evaluation.

4.1 Training Dataset Composition

The training set is composed of the training portions of PROSIT and Vital Videos. According to the table information in the text, the specific composition is as follows:

• PROSIT (Internal): 128 participants, 5,700 clips, 16.8 hours.
• Vital Videos (EU): 589 participants, 2,238 clips, 9.3 hours.
• Vital Videos (Africa): 383 participants, 2,869 clips, 7.7 hours.
• Vital Videos (South Asia): 313 participants, 3,887 clips, 10.8 hours.
• Total: 1,413 participants, 14,694 clips, 44.6 hours.

The dataset maintains a good balance in demographic characteristics, including gender balance, and covers all six Fitzpatrick skin types. Physiological metric distributions are wide, covering a broad range of heart rate variability states.

4.2 Test Dataset Composition

The test set includes 422 unique individuals from four public and private datasets, totaling 1,081 video clips.

• Sources: PROSIT (Internal), Vital Videos (EU/Africa/South Asia), UBFC-Phys, UBFC-rPPG.
• Processing: All video clips are processed to durations of 20-60 seconds, suitable for time-domain HRV analysis.
• Quality Control: Samples with known label quality or synchronization issues (e.g., partial data from UBFC-Phys) are excluded to ensure fair and reliable benchmarking.

5. Model Architecture and Training Strategy

5.1 Model Architecture

VitalLens 2.0 is an end-to-end deep convolutional neural network.

• Input: Sequence of facial video frames.
• Output: Two continuous time-series signals—Pulse (PPG) waveform and Respiration (RESP) waveform.
• Downstream Metrics: Heart Rate (HR), Respiratory Rate (RR), and HRV metrics (such as SDNN, RMSSD, LF/HF) are derived from these waveforms using industry-standard signal processing techniques.
• Core Design: The architecture specifically focuses on minimizing signal noise and preserving the precise temporal location of systolic peaks.

5.2 Training and Validation Methods

• Strict Separation: Follows a strict participant separation method. 1,413 training individuals are used to optimize parameters, while an independent validation set is used to monitor overfitting and hyperparameter tuning.
• Baseline Comparison: For fair comparison, all learning-based baseline models (e.g., DeepPhys, MTTS-CAN, EfficientPhys) are retrained from scratch on the same 1,413-person training dataset.
• Internal Baselines:
  • VitalLens 1.0: Original model trained on a smaller dataset.
  • VitalLens 1.1: Original architecture retrained on the new large dataset.
  • This comparison helps isolate performance gains from architectural improvements versus data improvements.

5.3 HRV Calculation Flow

Estimating HRV metrics from the predicted PPG waveform follows a multi-stage signal processing flow:

1. Peak Detection: Identify valid peaks considering signal significance, width, and periodicity relative to rolling frequency estimates.
2. Filtering: Retain only high-confidence peaks.
3. IBI Calculation: Calculate Inter-Beat Intervals (IBIs) from the peak sequence.
4. Cleaning: Interpolate outlier intervals that exceed physiologically reasonable ranges (e.g., due to missed or false detections).
5. Metric Calculation: Calculate time-domain (SDNN, RMSSD) and frequency-domain (LF/HF) metrics from the cleaned IBI sequence.

6. Experimental Results and Evaluation

6.1 Evaluation Metrics

• Waveform Level: Pearson correlation coefficient (r) and Signal-to-Noise Ratio (SNR).
• Physiological Metric Level: Mean Absolute Error (MAE), for HR, RR, SDNN, RMSSD, LF/HF.
• Reporting Strategy: First, average the metrics of all clips belonging to a single individual, and then average these individual scores. This prevents individuals with more video clips from disproportionately influencing the aggregated results.

6.2 Performance

On the combined test set containing 422 individuals, VitalLens 2.0 significantly outperforms other methods on almost all metrics, establishing a new State-of-the-Art (SOTA) for learning-based rPPG models.

Key Performance Data (MAE Averaged per Individual):

• Heart Rate (HR): 1.57 bpm
• Respiratory Rate (RR): 1.08 bpm
• HRV-SDNN: 10.18 ms
• HRV-RMSSD: 16.45 ms

6.3 Comparative Analysis

VitalLens 2.0 significantly outperforms:

• Hand-crafted Algorithms: Such as POS, G, CHROM.
• Other Learning Methods: Such as DeepPhys, MTTS-CAN, EfficientPhys.
• Previous Generation Models: Significantly better than the VitalLens 1.0 architecture.

The description of Figure 1 in the text shows a visual comparison of the waveforms generated by VitalLens 2.0 against Ground Truth and other models (VitalLens 1.0, POS). The results show that VitalLens 2.0 achieves higher fidelity and can accurately reconstruct the precise timing of systolic peaks.

7. Summary and Outlook

VitalLens 2.0 represents a significant advancement in rPPG technology, successfully expanding the application scope from simple heart rate/respiratory rate monitoring to high-fidelity Heart Rate Variability (HRV) analysis. By combining a large-scale diverse dataset with an optimized temporal attention network architecture, the model achieves new performance benchmarks on multiple public and private datasets.

The model is now available to developers via the VitalLens API, enabling developers to integrate robust real-time HRV analysis into their applications. This progress lays a solid foundation for the widespread application of non-invasive health monitoring in mobile devices and telemedicine scenarios.