Neural-symbolic system for multimodal visual reasoning towards digital twin
Faced with the complexities of fusing heterogeneous multimodal visual data in digital twins,a novel neuro-symbolic approach for combining the analytical capabilities of deep learning with the structured reasoning of symbolic intelligence was proposed.This approach employed deep neural networks to analyze the visual data in real-time and supplemented autonomous management of complex reasoning processes by the knowledge and event-response rules stored in a symbolic system.To enhance the system's adaptability for the physical world changes,an augmented rea-soning mechanism integrating multimodal information with external knowledge was proposed.This mechanism ef-fectively consolidated real-time sensor data with information from historical knowledge bases to support more accu-rate and rational decision-making.The efficacy of the proposed method was demonstrated through a case study on the disassembly of retired lithium batteries,and its capability to achieve high accuracy in identifying and analyzing multimodal data was illustrated.Furthermore,the coherent and logical operational recommendations based on the reasoning capabilities were generated,which significantly improved disassembly efficiency and safety.
digital twinmulti-modalvisual reasoningneural-symbolic systemlithium battery disassembly
NETL
NSTL
万方数据
