Facial video-based heart rate measurement against irregular motion artifacts
Objective Heart rate(HR)is one of the most important physiological parameters that can reflect the physical and mental status of individuals.Various methods have been developed to estimate HR values using contact and noncontact sensors.The advantage of noncontact methods is that they provide a more comfortable and unobtrusive way to estimate HR and avoid discomfort or skin allergy caused by conventional contact methods.The pulse-induced subtle color variations of facial skins can be measured from consumer-level cameras.Thus,camera-based non-contact HR detection technology,also called remote photoplethysmograph(rPPG),has been widely used in the fields of mobile health monitoring,driving safety,and emotion awareness.The principle of camera-based rPPG measurement is similar to that of traditional PPG mea-surement,that is,the pulsatile blood propagating in cardiovascular systems changes blood volumes in microvascular tissue beds beneath skins with each heartbeat,thus producing a fluctuation.However,such technology is susceptible to motion artifacts due to weak amplitudes of the physiological parameter information it carries.For instance,subjects'heads may move involuntarily during interviews,presentations,and other socially stressful situations,thus degrading rPPG-based HR detection performance.Accordingly,this paper proposes a novel motion-robust rPPG method that combines nonnegative matrix factorization(NMF)and independent vector analysis(IVA),termed as NMF-IVA,to remove irregular motion arti-facts.Method First,the whole facial region of interest(RoI)is divided into several sub RoIs(SRoIs),among which three optimal SRoIs are selected based on three indicators:average light intensity,light intensity variation of a certain SRoI,and signal-to-noise ratio(SNR)of the green-channel signal derived from the SRoI.Afterwards,three green-channel time series are derived from the corresponding three optimal SRoIs.Second,the three channels of time series are detrended,bandpass filtered,and then sent to the proposed NMF-IVA as input.After the NMF-IVA operation,three source signals are extracted and then processed by power spectral density analysis.The one with the highest peak SNR and the corresponding dominant frequency falling within the interested HR range will be identified as the blood volume pulse(BVP)signal,whose dominant frequency is identified as that of the estimated HR.Result We compare the proposed NMF-IVA method with seven typical rPPG methods on two publicly available datasets(UBFC-RPPG and UBFC-PHYS)as well as one in-house dataset.On the UBFC-RPPG dataset,compared with the second-best performance of the single channel filtering(SCF)method,the proposed NMF-IVA achieves better performance,with an improved root mean square error(RMSE)of HR measurement by 1.39 beat per minute(bpm),an improved mean absolute error(MAE)by 1.25 bpm,and a higher Pearson's correlation coefficient(PCC)by 0.02.Although both the MAE and the RMSE achieved by the proposed NMF-IVA method are lower than those of deep learning-based methods,the PCC of the NMF-IVA is comparable to that of deep learning-based ones,which demonstrates the effectiveness of the proposed NMF-IVA method.As for the UBFC-PHYS data-set when compared with traditional rPPG methods,during the T1 condition,the performance of the proposed NMF-IVA method is better than that of the second-best SCF method,with an improved RMSE by 6.45 bpm,an improved MAE by 2.53 bpm,and a higher PCC by 0.18.When compared with deep learning-based ones,the proposed NMF-IVA method achieves the second-best performance.The performance improvement of the proposed NMF-IVA is most noticeable during the T2 condition on the UBFC-PHYS dataset.Specifically,when compared with the second-best performance of IVA,the above three metrices are improved by 16.42 bpm,9.91 bpm,and 0.64,respectively.As for the UBFC-PHYS dataset,when during the T3 condition,the best performance is still achieved by the proposed NMF-IVA method.When compared with the second-best performance of the independent component analysis method,the corresponding three metrices are improved by 8.54 bpm,6.14 bpm,and 0.37,respectively.The performance of the proposed NMF-IVA method can be comparable to that of deep learning-based ones both in T2 and T3 conditions.As for the in-house dataset,the proposed NMF-IVA method achieves better performance compared with the traditional methods,except for deep learning-based meth-ods.Conclusion The proposed NMF-IVA method achieves the best results on all the three datasets when compared with traditional rPPG methods,and the performance improvement is most noticeable during irregular motion artifact conditions involving head motions with large amplitudes.However,the performance of the proposed NMF-IVA method is slightly poorer than that of deep learning-based methods possibly because deep learning technology has excellent abilities in learn-ing and extracting effective features.However,sufficient training samples and generalization should be considered when adopting deep learning-based methods.In addition,before the high-quality BVP source is derived,upsampling is employed,which leads to a relatively large time consumption.In the future,the HR estimation performance and the upsampling rate should be traded off.The proposed NMF-IVA method has advantages in extracting regular signals.Thus,our study can provide a new solution for promoting the practical application ability of rPPG technology.
NETL
NSTL
万方数据
