首页|IBNR-RD: Intra-Block Neighborhood Relationship-Based Resemblance Detection for High-Performance Multi-Node Post-Deduplication

IBNR-RD: Intra-Block Neighborhood Relationship-Based Resemblance Detection for High-Performance Multi-Node Post-Deduplication

扫码查看
原文链接
  • NETL
  • NSTL
  • IEEE
Post-deduplication in traditional cloud environments primarily focuses on single-node, where delta compression is performed on the same deduplication node located on server side. However, with data explosion, the multi-node post-deduplication, also called global deduplication, has become a hot issue in research communities, which aims to simultaneously execute delta compression on data distributed across all nodes. Simply setting up single-node deduplication systems on multi-node environments would significantly affect storage utilization and incur secondary overhead from file migration. Nevertheless, existing global deduplication solutions suffer from lower data compression ratios and high computational overhead due to their resemblance detection's inherent limitations and overly coarse granularities. Similar blocks typically have high correlations between sub-blocks; inspired by this observation, we propose IBNR (Intra-Block Neighborhood Relationship-Based Resemblance Detection for High-Performance Multi-Node Post-Deduplication), which introduces a novel resemblance detection based on relationships between sub-blocks and determines the ownership of blocks in entry stage to achieve efficient global deduplication. Furthermore, the by-products of IBNR have shown powerful scalability by replacing internal resemblance detection scheme with existing solutions on practical workloads. Experimental results indicate that IBNR outperforms state-of-the-art solutions, achieving an average 1.99× data reduction ratio and varying degrees of improvement across other key metrics.

Feature extractionCloud computingFingerprint recognitionTrainingComputational modelingAccuracyTransformsProductionCostsComputer science

Dewen Zeng、Wenlong Tian、Tingting He、Ruixuan Li、Xuming Ye、Zhiyong Xu

展开 >

School of Computer Science and Technology, University of South China, Hengyang, China

College of Educational Science, Hengyang Normal University, Hengyang, China

School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China

Math and Computer Science, Suffolk University, Boston, MA, USA

展开 >

2025

10.1109/TCC.2024.3514784

IEEE transactions on cloud computing

IEEE transactions on cloud computing

ISSN：
年,卷(期)：2025.13(1)
  • 45