山东大学学报(工学版)2024,Vol.54Issue(5) :101-110.DOI:10.6040/j.issn.1672-3961.0.2023.271

基于分解式Transformer的联邦长期时间序列预测算法

Federated long-term time series forecasting algorithm based on decomposed Transformer

刘冬兰 刘新 刘家乐 赵鹏 常英贤 王睿 姚洪磊 罗昕
山东大学学报(工学版)2024,Vol.54Issue(5) :101-110.DOI:10.6040/j.issn.1672-3961.0.2023.271

基于分解式Transformer的联邦长期时间序列预测算法

Federated long-term time series forecasting algorithm based on decomposed Transformer

刘冬兰 1刘新 1刘家乐 2赵鹏 3常英贤 3王睿 1姚洪磊 1罗昕2
扫码查看

作者信息

  • 1. 国网山东省电力公司电力科学研究院,山东 济南 250003;山东省智能电网技术创新中心,山东 济南 250003
  • 2. 山东大学软件学院,山东 济南 250101
  • 3. 国网山东省电力公司,山东 济南 250001
  • 折叠

摘要

为解决基于Transformer的方法存在计算成本高和无法捕捉时间序列总体趋势的问题,将Transformer与季节性趋势分解法相结合,提出基于分解式Transformer的联邦长期时间序列预测算法,其中分解方法用于捕捉时间序列的全局概况.在实际场景中,时间序列数据来自多个不同客户端.考虑数据隐私问题,利用联邦学习从多个客户端获得整体最优预测模型,采用基于局部锐度感知最小化的优化器提高全局模型的泛化性.与先进的方法相比,该方法在 4 个基准数据集的多变量和单变量时间序列预测任务中都有改进,在用电负荷(electricity consuming load,ECL)数据集上性能最高可提升 26.9%.试验结果充分表明季节性趋势分解法与局部锐度感知最小化的优化器在长期时间序列预测任务上的有效性.

Abstract

To address the issues of high computational costs and inability to capture the overall trend of time series using Transformer based method,a combined approach of Transformer and seasonal trend decomposition was proposed.A novel federated long-term time series forecasting algorithm based on decomposed Transformer was introduced,where the decomposition method was employed to capture the global overview of time series.In practical scenarios,time series data originated from multiple different clients.Con-sidering data privacy concerns,a federated learning approach was utilized to obtain an overall optimal forecasting model from multi-ple clients,employing an optimizer based on locally sharpness-aware minimization(SAM)to improve the generalization of the global model.Compared with advanced methods,improvements were observed across multivariate and univariate time series forecas-ting tasks on four benchmark datasets,with the highest performance enhancement reaching 26.9%on the electricity consuming load(ECL)dataset.Experimental results strongly indicated the effectiveness of seasonal trend decomposition and the SAM optimizer in long-term time series forecasting tasks.

关键词

隐私保护/联邦学习/长期预测/模型泛化/Transformer

Key words

privacy protection/federated learning/long-term forecasting/model generalization/Transformer

引用本文复制引用

基金项目

国网山东省电力公司科技资助项目(520626220018)

出版年

2024
山东大学学报(工学版)
山东大学

山东大学学报(工学版)

CSTPCD北大核心
影响因子:0.634
ISSN:1672-3961
段落导航相关论文