首页|Cross-Modal Graph Semantic Communication Assisted by Generative Al in the Metaverse for 6G

Cross-Modal Graph Semantic Communication Assisted by Generative Al in the Metaverse for 6G

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
Recently,the development of the Metaverse has become a frontier spotlight,which is an important demonstration of the integration innovation of advanced technologies in the Internet.Moreover,artificial intelligence(AI)and 6G communications will be widely used in our daily lives.However,the effective interactions with the representations of multimodal data among users via 6G communications is the main challenge in the Metaverse.In this work,we introduce an intelligent cross-modal graph semantic communication approach based on generative Al and 3-dimensional(3D)point clouds to improve the diversity of multimodal representations in the Metaverse.Using a graph neural network,multimodal data can be recorded by key semantic features related to the real scenarios.Then,we compress the semantic features using a graph transformer encoder at the transmitter,which can extract the semantic representations through the cross-modal attention mechanisms.Next,we leverage a graph semantic validation mechanism to guarantee the exactness of the overall data at the receiver.Furthermore,we adopt generative Al to regenerate multimodal data in virtual scenarios.Simultaneously,a novel 3D generative reconstruction network is constructed from the 3D point clouds,which can transfer the data from images to 3D models,and we infer the multimodal data into the 3D models to increase realism in virtual scenarios.Finally,the experiment results demonstrate that cross-modal graph semantic communication,assisted by generative Al,has substantial potential for enhancing user interactions in the 6G communications and Metaverse.

Mingkai Chen、Minghao Liu、Congyan Wang、Xingnuo Song、Zhe Zhang、Yannan Xie、Lei Wang

展开 >

Key Laboratory of Broadband Wireless Communication and Sensor Network Technology,Nanjing University of Posts and Telecommunications,Nanjing 210003,China

State Key Laboratory of Organic Electronics and Information Displays & Institute of Advanced Materials(IAM),Nanjing University of Posts and Telecommunications,Nanjing 210023,China

National Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaNational Natural Science Foundation of ChinaKey R and D Program of Jiangsu Province Key project and topicsKey R and D Program of Jiangsu Province Key project and topicsNatural Science Foundation of Jiangsu ProvinceNatural Science Research Startup Foundation of Recruiting Talents of Nanjing University of Posts and TelecommunicationsKey Project ofNatural Science Foundation of Jiangsu Provincemajor projects of the Natural Science Foundation of the Jiangsu Higher Education institutions

62001246622310176220127762071255BE2021095BE2023035BK20220390NY221011BE202308720KJA510009

2024

10.34133/research.0342

研究(英文)

研究(英文)

CSTPCD
ISSN：
年,卷(期)：2024.2024(2)
  • 1