Analyzing intrusion intentions and penetration behaviors from the attackers'perspective is of great significance for guiding network security defense.However,most existing penetration paths are constructed based on the instantaneous network environment,resulting in reduced reference value.Aiming at this problem,this paper proposes an optimal penetration path genera-tion method based on maximum entropy reinforcement learning,which can capture the approximate optimal behavior of multiple modes in the form of exploration under dynamic network environments.Firstly,the penetration process is modeled according to the attack graph and the vulnerability score,and the threat degree of the penetration behavior is described by quantifying the at-tack benefits.Then,considering the complexity of the intrusion behavior,a soft Q-learning method based on the maximum entro-py model is developed.The stability of the penetration path is ensured by controlling the entropy value and the importance of the reward.Finally,the method is applied to a dynamic environment to generate a highly available penetration path.Simulation experi-mental results show that,compared with the existing baseline methods based on reinforcement learning,the proposed method has more robust environmental adaptability and can generate higher-yielding penetration paths at a lower cost.
关键词
最大熵强化学习/攻击图/Soft/Q-学习/渗透路径
Key words
Maximum entropy reinforcement learning/Attack graph/Soft Q-learning/Penetration path
维普
万方数据