Design and Upgrade Practice of Urban Rail Big Data Platform Based on"Data Lakehouse"Technology
To explore the optimal scheme for the construction and upgrading of the big data platform in the urban rail transit industry,this study takes the urban rail big data platform as the research object.Our study starts from the development process of the urban rail big data platform,sorts out the three stages of the development of the urban rail big data platform and analyzes the technology and advantages and disadvantages of the big data platform at each stage.Then it focuses on summarizing the advantages of"Data lake and Warehouse integration,stream processing and batch processing integration,OLTP+OLAP,multiple loads"and other advantages of the"Data Lakehouse"big data technology in the current stage,and studies the key points of the architecture upgrade and transformation design of the big data platform based on this technology.The technology was verified in the upgradation and transformation of the big data platform of the Beijing Metro Data Center.The application shows that the"Data lakehouse"big data platform technology combines the advantages of low cost of data lake and high performance of data warehouse,solves the shortcomings of the original big data platform in performance,capacity and multi-purpose support,and provides new solutions for the construction and upgradation of big data platforms in the urban rail industry.
urban rail transitbig data platformupgradingdata Lakehousestream processing and batch processing integrationdata warehousedata lake
万方数据
