首页|基于拓展α-rank的多智能体策略评估方法在能源管理中的应用

基于拓展α-rank的多智能体策略评估方法在能源管理中的应用

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
随着碳达峰、碳中和政策的制定与实施,电网新能源化成为了主流趋势.然而,配电网中光伏装置数量的增加已经给分布式配电网系统带来巨大的有源电压调控压力,使得传统电压调节模式难以适应新能源化电网系统.基于多智能体强化学习的智能控制策略可通过智能逆变器和其他智能建筑能源管理系统(楼宇微网)缓解这些问题.为了获得楼宇微网的最佳能源管理策略,并满足楼宇用户的舒适度和能源需求,本文提出两种大规模多智能体策略评估方法,将能源管理问题转化为一般和博弈,同时优化了系统和楼宇用户两个层面的收益.α-rank算法虽然可解决一般和博弈,并在理论上保证策略排名的可靠性,但其受到策略交互中的采样复杂度限制,难以应用于实际电力系统.通过引入张量补全拓展α-rank算法,本文提出一种新的评估算法TcEval,以降低交互中的采样复杂性.此外,考虑到实际场景中普遍存在的噪声问题,本文建立了基于领域知识的噪声处理模型来计算策略收益,提出了针对噪声场景的TcEval-AS算法.多组基于实际数据的能源管理案例实验说明,本文提出的两种评估算法相较于现有方法(RG-UCB和α-IG)大幅度降低了策略评估中采样复杂度.最后,用实际数据验证了所提算法的有效性.
Multi-agent evaluation for energy management by practically scaling α-rank
Currently,decarbonization has become an emerging trend in the power system arena.However,the increasing number of photovoltaic units distributed into a distribution network may result in voltage issues,providing challenges for voltage regulation across a large-scale power grid network.Reinforcement learning based intelligent control of smart inverters and other smart building energy management(EM)systems can be leveraged to alleviate these issues.To achieve the best EM strategy for building microgrids in a power system,this paper presents two large-scale multi-agent strategy evaluation methods to preserve building occupants'comfort while pursuing system-level objectives.The EM problem is formulated as a general-sum game to optimize the benefits at both the system and building levels.The α-rank algorithm can solve the general-sum game and guarantee the ranking theoretically,but it is limited by the interaction complexity and hardly applies to the practical power system.A new evaluation algorithm(TcEval)is proposed by practically scaling the α-rank algorithm through a tensor complement to reduce the interaction complexity.Then,considering the noise prevalent in practice,a noise processing model with domain knowledge is built to calculate the strategy payoffs,and thus the TcEval-AS algorithm is proposed when noise exists.Both evaluation algorithms developed in this paper greatly reduce the interaction complexity compared with existing approaches,including ResponseGraphUCB(RG-UCB)and αInformationGain(α-IG).Finally,the effectiveness of the proposed algorithms is verified in the EM case with realistic data.

Energy managementMulti-agent deep reinforcement learningStrategy evaluationPower grid system

孙祎芸、张森林、刘妹琴、郑荣濠、董山玲、兰旭光

展开 >

浙江大学工业控制技术国家重点实验室,中国 杭州市,310027

浙江大学电气工程学院,中国 杭州市,310027

西安交通大学人机混合增强智能全国重点实验室,中国西安市,710049

能源管理 多智能体深度强化学习 策略评估 配电网系统

National Key R&D Program of ChinaZhejiang Provincial Natural Science Foundation of ChinaFundamental Research Funds for the Central Universities,China

2021ZD0112700LZ22F030006xtr072022001

2024

10.1631/FITEE.2300438

信息与电子工程前沿(英文)
浙江大学

信息与电子工程前沿(英文)

CSTPCD
影响因子：0.371
ISSN：2095-9184
年,卷(期)：2024.25(7)
  • 1