Voice navigation anti-interference system based on multimodal lip state recognition
To solve the problem that the existing in-vehicle voice navigation devices were susceptible to interference from the noise both inside and outside vehicle and could not accurately determine the source of sound signals,the voice navigation anti-interference system based on lip state recognition was proposed.Using a camera to perform real-time recognition of the driver lip state,the start and end points of the driver voice signal were accurately determined,and the activation and deactivation of the voice navigation input signal were controlled for enhancing the driver control over the voice navigation and reducing the interference from the noise inside and outside vehicle.To accurately assess the accuracy and robustness of lip state recognition,the multimodal lip state recognition network based on key point-appearance short-term feature fusion was proposed.The experiment of validating the effectiveness of key point short-term features,the ablation experiment of multimodal feature fusion in lip state recognition and the voice navigation anti-interference tests in both simulated laboratory environments and real in-vehicle environments were conducted.The results show that the proposed key point short-term feature operator can enhance the representation ability of lip state changes by more than 14％.The key point-appearance fusion lip state recognition network improves the recognition accuracy by 8.98％through feature complementation.The voice navigation anti-interference system based on this network exhibits high accuracy of 92.6％and good real-time performance with detection speed of 35 F/s.The interference from the noise inside and outside vehicle on the driver voice control authority can be effectively reduced even under the significant head pose changes of more than 70 degrees to the left or right,which demonstrates high robustness.
万方数据
维普
