Research on the infrastructure of real-time data warehouse based on ClickHouse
With the advancement of mobile internet technology and the increasing participation of users in online shopping,e-commerce enterprises are harvesting a large amount of user behavior logs and business data every day,even every hour.Tradi-tional real-time computing systems are no longer able to meet the online analysis and real-time statistics of these logs and business data.Inspired by this requirement,a basic architecture for real-time processing of user behavior data is studied based on the lay-ered design concept,aiming to improve computational reusability and reduce development costs by precipitating intermediate re-sults when facing a large amount of real-time computing.This architecture adopts the real-time analytical columnar database Click-House and Flink real-time streaming processing framework as the core technologies,and obtains data at the day,minute,second,and even sub second levels through real-time computing,making it easy for enterprises to quickly respond and adjust to business,and meeting the real-time computing needs of the new era.
ClickHousereal-time computing architectureFlinkbig data
万方数据
维普
