首页|基于强化学习算法的两阶段无人船协同调度方法

基于强化学习算法的两阶段无人船协同调度方法

扫码查看
疫情过后会迎来经济增长,集装箱货运量会显著增加,现有码头陆地侧设施对集装箱处理能力越发不足.因此,为提高集装箱在码头间的转运效率,本文提出了两阶段调度方法,来驱动无人船在码头水域侧运输集装箱.在第一阶段中,考虑到无人船再充电模式、时间窗、协调泊位等限制,建立以作业延误最小为目标的任务规划模型,减少无人船延误所带来的惩罚;再构建以执行成本最小为目标的任务规划模型,获得无路径冲突运输方案.在第二阶段中,提出无人船路径跟踪控制方法.考虑舵角饱和约束限制,设计了模型预测控制器来平稳地实现无人船路径控制.最后,提出了 MARL算法,该算法可结合历史训练数据快速获得求解问题最优解.仿真结果表明:提出的调度方法能够获得低成本的无人船运输方案,并且提出的算法对大规模问题仍具有较好求解表现.
Two-stage unmanned vessels co-scheduling method based on the reinforcement learning algorithm
After the pandemic,there will be economic growth,and the volume of container shipping will significantly increase,but the existing port and land-side facilities are increasingly inadequate to handle the container capacity.Therefore,to improve the efficiency of container transfer between ports,this paper proposes a two-stage scheduling method to drive unmanned vessels for container transportation in the port waterside.In the first stage,taking into ac-count the recharging mode of unmanned vessels,time windows,coordinated berths,and other constraints,a task planning model is established with the goal of minimizing operation delays,reducing the penalties caused by unmanned vessel delays.Another task planning model is then constructed with the objective of minimizing execution costs to obtain a collision-free transport plan.In the second stage,an unmanned vessel path tracking control method is proposed.Consid-ering rudder angle saturation constraints,a model predictive controller is designed to smoothly achieve unmanned vessel path control.Finally,an MARL algorithm is introduced,which,in conjunction with historical training data,quickly obtains optimal solutions to problem-solving.Simulation results show that the proposed scheduling method can obtain low-cost unmanned ves-sel transport plans,and the introduced algorithm still demonstrates good performance in solving large-scale problems.

USVstwo-stagescheduling methodreinforcement learning

郭兴海、李紫萌、计明军、贾福生、胡玉真

展开 >

哈尔滨工程大学经济管理学院,哈尔滨 150001

哈尔滨工程大学国家大学科技园,哈尔滨 150001

大连海事大学交通运输工程学院,大连 116024

中国青岛港务集团,青岛 266706

展开 >

无人船 两阶段 调度方法 强化学习

2024

系统工程理论与实践
中国系统工程学会

系统工程理论与实践

CSTPCDCSSCI北大核心
影响因子:1.575
ISSN:1000-6788
年,卷(期):2024.44(10)