首页|An innovative heterogeneous transfer learning framework to enhance the scalability of deep reinforcement learning controllers in buildings with integrated energy systems

An innovative heterogeneous transfer learning framework to enhance the scalability of deep reinforcement learning controllers in buildings with integrated energy systems

扫码查看
原文链接
  • NETL
  • NSTL
  • 万方数据
Deep Reinforcement Learning(DRL)-based control shows enhanced performance in the management of integrated energy systems when compared with Rule-Based Controllers(RBCs),but it still lacks scalability and generalisation due to the necessity of using tailored models for the training process.Transfer Learning(TL)is a potential solution to address this limitation.However,existing TL applications in building control have been mostly tested among buildings with similar features,not addressing the need to scale up advanced control in real-world scenarios with diverse energy systems.This paper assesses the performance of an online heterogeneous TL strategy,comparing it with RBC and offline and online DRL controllers in a simulation setup using EnergyPlus and Python.The study tests the transfer in both transductive and inductive settings of a DRL policy designed to manage a chiller coupled with a Thermal Energy Storage(TES).The control policy is pre-trained on a source building and transferred to various target buildings characterised by an integrated energy system including photovoltaic and battery energy storage systems,different building envelope features,occupancy schedule and boundary conditions(e.g.,weather and price signal).The TL approach incorporates model slicing,imitation learning and fine-tuning to handle diverse state spaces and reward functions between source and target buildings.Results show that the proposed methodology leads to a reduction of 10％in electricity cost and between 10％and 40％in the mean value of the daily average temperature violation rate compared to RBC and online DRL controllers.Moreover,online TL maximises self-sufficiency and self-consumption by 9％and 11％with respect to RBC.Conversely,online TL achieves worse performance compared to offline DRL in either transductive or inductive settings.However,offline Deep Reinforcement Learning(DRL)agents should be trained at least for 15 episodes to reach the same level of performance as the online TL.Therefore,the proposed online TL methodology is effective,completely model-free and it can be directly implemented in real buildings with satisfying performance.

transfer learningreinforcement learningbuilding controlbuilding energy management

Davide Coraci、Silvio Brandi、Tianzhen Hong、Alfonso Capozzoli

展开 >

Politecnico di Torino,Department of Energy,TEBE research group,BAEDA Lab.,Corso Duca degli Abruzzi 24,Torino,10129,Italy

Building Technology and Urban Systems Division,Lawrence Berkeley National Laboratory,One Cyclotron Road,Berkeley,CA 94720,USA

project NODESMUR-M4C2 1.5 of PNRR funded by the European Union-NextGenerationEUOpen access funding provided by Politecnico di Torino within the CRUI-CARE Agreement

ECS00000036

2024

10.1007/s12273-024-1109-6

建筑模拟(英文版)

建筑模拟(英文版)

EI
ISSN：1996-3599
年,卷(期)：2024.17(5)