The exponential growth of news data has presented a significant challenge to efficiently storing and managing news.To address the bottleneck issues of news text storage and access in the separation mode of storage and computation,we proposed an optimization method for news text storage based on Alluxio,a data orchestration platform.Firstly,Alluxio was utilized to cache news texts across various storage systems and provided a unified namespace,enhancing data access for computational applications.Secondly,to overcome performance limitations in reading and accessing remotely stored news text,we incorporated directory aggregation for merging the news text and emploied the Minimum Perfect Hash algorithm to index metadata associated with the news text.This enables rapid retrieval of relevant information from the stored news texts.
NETL
NSTL
万方数据
