3D Gaze Estimation by Bidirectional Fusion of CNN and Transformer
To address the issue of low accuracy and susceptibility to interference from external factors in unconstrained environments,a convolution and attention double-branch parallel feature cross-fusion gaze estimation method is proposed to enhance feature fusion effectiveness and network performance.Firstly,the Mobile-Former network is enhanced by introducing a linear attention mechanism and partial convolution.This effectively improves the feature extraction capability while reducing computing costs.Additionally,a branch of the ResNet50 head pose feature estimation network,pre-trained on the 300W-LP dataset,is added to enhance gaze estimation accuracy.A Sigmoid function is used as a gating unit to screen effective features.Finally,facial images are inputted into the neural network for feature extraction and fusion,and the 3D gaze estimation direction is outputted.The model is evaluated on the MPIIFaceGaze and Gaze360 datasets,and the average angle error of the proposed method is 3.70°and 10.82°,respectively.The network model is verified to accurately estimate the 3D gaze direction and reduce computational complexity compared to other mainstream 3D gaze estimation methods.
3D gaze estimationparallel structurebidirectional fusionpartial convolutionlinear attention mechanism
NETL
NSTL
万方数据
