Efficient Vertical Federated Learning Based on Embedding and Gradient Bidirectional Compression
Vertical federated learning improves the value of data utilization by combining local data features from multiple parties and jointly training the target model without leaking data privacy.It has received widespread attention from companies and institutions in the industry.During the training process,the intermediate embeddings uploaded by clients and the gradients returned by the server require a huge amount of communication,and thus the communication cost becomes a key bottleneck limiting the practical application of vertical federated learning.Consequently,current research focuses on designing effective algorithms to reduce the communication amount and improve communication efficiency.To improve the communication efficiency of vertical federated learning,this study proposes an efficient compression algorithm based on embedding and gradient bidirectional compression.For the embedding representation uploaded by the client,an improved sparsification method combined with a cache reuse mechanism is employed.For the gradient information distributed by the server,a mechanism combining discrete quantization and Huffman coding is used.Experimental results show that the proposed algorithm can reduce the communication volume by about 85%,improve communication efficiency,and reduce the overall training time while maintaining almost the same accuracy as the uncompressed scenario.
NETL
NSTL
万方数据
