Currently,the meteorological big data cloud platform exclusively serves meteorological Intranet users.In order to fur-ther expand the platform's capability for external services and meet the extensive demands of meteorological industry users for the storage and sharing of massive meteorological data applications,a secure and efficient meteorological big data storage and ser-vice system is designed based on key technologies such as distributed storage and data service interfaces.This system is imple-mented in the Unidirectional Network Security Isolation Zone(Demilitarized Zone,DMZ).For structured meteorological data,an optimized data synchronization method based on the Database Binlog in a distributed database is researched and designed.For unstructured meteorological data,an optimized data synchronization method combining the Database Binlog and ETL mecha-nisms is researched and designed to achieve real-time second-level unidirectional synchronization of massive meteorological data between the meteorological intranet big data cloud platform and the DMZ area.The method proposed in this paper addresses issues such as low data volume,poor sharing service capabilities,and low security for meteorological data in industry user ser-vices.Application practices demonstrate that the data synchronization technology achieves synchronization delays within seconds,maintains good data consistency,and ensures the real-time and accurate external service of the meteorological big data cloud platform.
关键词
数据同步/Binlog/ETL/气象大数据/DMZ区
Key words
data synchronization/Binlog/ETL/meteorological big data/DMZ zone
维普
万方数据