Multimodal Fusion Detection Method of Tomato Fruit Based on Phase and Hyperspectral
Tomatoes are highly nutritious and popular worldwide as both a vegetable and a fruit.With the improvement of consumers' requirements for food quality and the access standards of tomato products in various countries,tomato quality control has received more and more attention.Traditional manual sorting operations are time-consuming,laborious and inefficient,so it is necessary to develop fast and accurate inspection technology.With the development of machine vision and neural network technologies,automated agricultural product defect detection methods have been widely studied and applied,providing new ideas and methods for agricultural product quality inspection.However,tomatoes and other agricultural products are affected by genetic factors and growth environment,and their external three-dimensional geometric forms and internal physiological information are complex and different,and a single type of detection method can only detect specific defects,so more dimensional detection methods are needed to detect the quality of tomatoes.In view of the needs of automatic detection of tomato quality and the shortcomings of traditional hyperspectral imaging technology in the non-destructive testing of agricultural products,such as incomplete information characterization and spectral reflectance distribution affected by morphology,this work adopts the idea of fusion detection of structured light imaging and hyperspectral imaging,and designs and builds a detection device that can realize the non-destructive diagnosis of the three-dimensional morphology of the appearance of the sample and the internal physiological information in the same imaging room.Based on the deep learning technology to obtain the three-dimensional topography of the sample from the projection fringes,a data fusion algorithm with simple operation and low computational cost was designed to register the heterologous images collected by different sensors,and the multi-dimensional information of the appearance and internal physiological state of the sample was characterized in real time by a single image through the fusion of the three-dimensional topography information and the hyperspectral image.Based on the principle of structured light imaging and deep learning technology,an easy-to-operate and fast automatic three-dimensional topography reconstruction algorithm of samples and a method of matching and fusing spectral distribution data were proposed,and the corresponding detection device was built.For the three-dimensional surface topography of the measured object,based on the fringe imaging principle of monocular camera,the surface height information is mapped through the pixel semantic information output by the semantic segmentation network model.The matching and fusion of the physical morphology information and the biochemical composition information of the analyte were carried out,and the two heterologous images were registered and evaluated based on the line feature fitting of the standard reference object.The edge extraction algorithm,curve fitting algorithm and line feature fusion method are used to recover the spatial position and pixel area of the reference object,and then the spatial position of the object to be tested is restored as the registration element,and the rapid registration of heterologous images is realized through simple operation,and the real-time registration fusion of three-dimensional mapping and hyperspectral images is realized.Taking tomato fruit as an example,the test was carried out by the detection device.For samples with a diameter of 4～8 cm,the trained network model can predict the three-dimensional height distribution within 0.75 seconds,and the diameter and maximum height errors are within 4%.In this paper,the morphology distribution is directly extracted based on a single structured light fringe map,and the heterologous image registration is carried out based on point-line features,and the simultaneous acquisition and fusion characterization of multi-dimensional information around the physical and chemical information of the sample can provide a reference for overcoming the problem of insufficient evaluation index of a single type of image,and realize the simultaneous detection,analysis and characterization of the surface morphology and shallow chemical composition of the analyte.In this work,the feasibility and effectiveness of the proposed method of fusion of three-dimensional morphology image and spectral image from heterologous sensor are verified by experiments,which can provide data and technical reference for solving the problem of insufficient evaluation indicators in the process of fruit quality detection under a single sensor.
Hyperspectral imagingTopography reconstructionDeep learningImage fusionMulti-dimensional information representation
万方数据
维普
