KDPhys: An Attention Guided 3D to 2D Knowledge Distillation for Real-time Video-Based Physiological Measurement

Reading Notes

Personal reading notes. Original paper: arXiv:2601.00714v1.

arxiv_id: 2601.00714v1

Paper Deep Dive: KDPhys - Attention-Guided 3D to 2D Knowledge Distillation for Real-Time Video Physiological Measurement

1. Research Background and Motivation

With the surge in demand for telehealth and remote health monitoring, especially during the SARS-CoV-2 pandemic, camera-based non-contact physiological monitoring technologies have received widespread attention. Remote Photoplethysmography (rPPG) extracts physiological signals such as heart rate by capturing subtle changes in skin optical properties that reflect pulsatile blood volume changes.

Although existing deep learning methods (such as DeepPhys, EfficientPhys, etc.) perform excellently in rPPG signal extraction, they still face a trade-off between efficiency and accuracy:

• 2D CNN Models: High computational efficiency, suitable for real-time applications, but struggle to effectively capture global temporal information across frames.
• 3D CNN Models: Capable of utilizing global spatiotemporal information from video sequences with higher accuracy, but suffer from high computational complexity and long inference times, making deployment on edge devices difficult.

This paper proposes a framework named KDPhys aimed at resolving the above contradiction. The core idea is to utilize Knowledge Distillation (KD) technology to transfer global spatiotemporal features learned by a 3D CNN teacher model to a lightweight 2D CNN student model. To the authors’ knowledge, this is the first application of knowledge distillation technology in the rPPG field.

2. Core Method: KDPhys Framework

The overall training and inference flow of KDPhys is shown below. Through the knowledge distillation mechanism, this framework enables the lightweight student model to inherit the powerful feature extraction capabilities of the teacher model while maintaining low computational costs.

Caption: Overall framework of the KDPhys method. Left figure (a) shows the training mode with knowledge distillation, right figure (b) shows the inference process. Input is frame difference of consecutive frames. Attention Feature Distillation (AFD) loss promotes feature knowledge transfer from teacher to student model, while DILATE loss guides the student model to align with the ground truth PPG signal. From PDF page 3, corresponding to Paper Figure 1.

The main components of the framework include:

1. Input Processing: The input is the frame difference of consecutive frames, which helps capture motion information.
2. Teacher Model: Based on 3D CNN, responsible for extracting global spatiotemporal representations.
3. Student Model: Based on 2D CNN, with a lighter structure, aimed at real-time inference.
4. Loss Function: Combines Feature Distillation Loss (AFD Loss) and Signal Regression Loss (DILATE Loss), together constituting the Total Loss.

3. Network Architecture and Knowledge Distillation Strategy

To achieve compatibility between the 3D teacher model and the 2D student model, the authors made specific modifications to the architecture. 3D spatiotemporal feature representations are projected onto the 2D plane of the student network to align internal representations.

Caption: Details of KDPhys network architecture and modifications. Shows feature alignment and distillation process between 3D teacher model and 2D student model. From PDF page 3, corresponding to Paper Figure 2.

Key Architecture Design:

• Feature Alignment: Project 3D features onto the 2D plane to ensure the student model can understand the teacher model’s feature space.
• Regression Head Design: The student model does not use fully connected layers to regress 2D features to 1D PPG signals; instead, it employs ConvTranspose layers combined with TSM (Temporal Shift Module) and Adaptive Average Pooling 2D (AAP) modules. The AAP module reduces computational complexity by averaging feature map values while retaining key information required for the regression task.
• Attention Mechanism: Attention masks are integrated after every layer of both the teacher and student models to extract salient spatial features.

Attention Feature Distillation (AFD): Traditional distillation methods usually assign equal weights to all features of the teacher model. KDPhys adopts Attention Feature Distillation (AFD) technology, calculating channel weights to prioritize distilling important features. Experiments show that compared to basic KD methods, AFD reduces MAE by 13% and RMSE by 20%.

4. Loss Function Design: DILATE

When predicting PPG signals, detecting temporal changes is as important as accurately predicting the signal shape. For non-stationary physiological signals (such as PPG, ECG), the authors introduced the DILATE Loss Function (DIstortion Loss including shApe and TimE).

• Function: The DILATE loss function penalizes both shape errors and temporal localization errors simultaneously.
• Advantage: Compared to commonly used Mean Squared Error (MSE) or Pearson loss functions, DILATE handles the waveform morphology and temporal alignment issues of PPG signals better.
• Effect: Using DILATE loss compared to MSE loss reduces MAE by 46%.

5. Experimental Results and Analysis

The authors conducted extensive qualitative and quantitative evaluations on three benchmark datasets. Experimental results demonstrate the advantages of KDPhys in terms of accuracy, efficiency, and robustness. The following images show some experimental results and comparative analysis (from PDF page 8).

Caption: Visualization of experimental results 1. From PDF page 8.

Caption: Visualization of experimental results 2. From PDF page 8.

Caption: Visualization of experimental results 3. From PDF page 8.

Caption: Visualization of experimental results 4. From PDF page 8.

Key Performance Metrics:

• Number of Parameters: Only 0.23M parameters, approximately half of existing methods.
• Inference Speed: 56.67% faster than existing methods.
• Accuracy: The average MAE across all datasets is 1.78 bpm, a reduction of 18.15% compared to state-of-the-art methods.
• Robustness: Additional experiments under different environmental conditions and activity scenarios prove the model’s adaptability and robustness.

6. Summary and Outlook

The KDPhys framework proposed in this paper successfully transfers the global spatiotemporal perception capabilities of 3D CNNs to lightweight 2D CNN models, achieving a balance between accuracy and efficiency for real-time video physiological measurement.

Summary of Major Contributions:

1. Innovation: First application of knowledge distillation technology in the rPPG field, achieving 3D to 2D global temporal relationship distillation.
2. Attention Distillation: Introduced Attention Feature Distillation (AFD), significantly improving the effectiveness of feature transfer.
3. Specialized Loss Function: Proposed the DILATE loss function, effectively solving the joint optimization problem of PPG signal shape and timing.

This research provides strong technical support for real-time non-contact health monitoring on edge devices. Future work can further explore applications in more physiological parameters (such as blood oxygen, respiration rate) and complex dynamic scenarios.