Global Top-K Frequent Flow Measurement for Continuous Periods in Distributed Networks
In distributed networks,the measurment of the top-K frequent flows is crucial for applications like resource allocation and security monitoring.Existing works on top-K frequent flow measurement have limitations such as being unsuitable for dis-tributed network traffic measurement or only considering single time periods.To address these problems,this paper proposes a scheme for measuring global top-K frequent flows over continuous time periods in distributed networks.This involves deploying compact probabilistic data structures at distributed nodes to record network flow information.At the end of each time period,dis-tributed nodes send necessary information to a central node,which aggregates this to identify the global top-K frequent flows from the start of measurement to the current time period.Considering that each flow may appear at one or multiple measurement nodes,different methods are used to reduce transmission overhead.For flows appearing at a single node,a method of transmitting segmented minimum values is used to obtain a threshold.Experiments show that this method reduces the transmission overhead by over 50％compared to full transmission.For flows appearing at multiple nodes,a multi-stage error-free processing method and a single-stage fast processing method are proposed,catering to scenarios that cannot tolerate errors and actual high-speed network traffic,respectively.Compared to using existing single-period methods in each time period,experimental performance of transmis-sion overhead reduced by two orders of magnitude.Finally,a method using historical average increment information to reduce communication delay is also proposed,and experimental results show that it effectively reduces the average relative error of con-straint information.
Traffic measurementtop-K frequent flowsDistributed networkContinuous time periodsketch
NETL
NSTL
万方数据
