物流作为现代经济的重要组成部分，在国民经济和社会发展中发挥着重要作用。物流中的三维装箱问题(Three-dimensional bin packing problem，3D-BPP)是提高物流运作效率必须解决的关键难题之一。深度强化学习(Deep rein-forcement learning，DRL)具有强大的学习与决策能力，基于DRL的三维装箱方法(Three-dimensional bin packing method based on DRL，DRL-3DBP)已成为智能物流领域的研究热点之一。现有DRL-3DBP面对大尺寸容器3D-BPP时难以达成动作空间、计算复杂性与探索能力之间的平衡。为此，提出一种四向协同装箱(Four directional cooperative packing，FDCP)方法:两阶段策略网络接收旋转后的容器状态，生成4个方向的装箱策略;根据由4个策略采样而得的动作更新对应的4个状态，选取其中价值最大的对应动作为装箱动作。FDCP在压缩动作空间、减小计算复杂性的同时，鼓励智能体对4个方向合理装箱位置的探索。实验结果表明，FDCP在100 × 100大尺寸容器以及20、30、50箱子数量的装箱问题上实现了1。2％～2。9％的空间利用率提升。
A Four Directional Cooperative Three-dimensional Packing Method Based on Deep Reinforcement Learning
As an important part of the modern economy,logistics plays an important role in the national economy and social development.The three-dimensional bin packing problem(3D-BPP)in logistics is one of the key prob-lems that must be solved to improve the efficiency of logistics operations.Deep reinforcement learning(DRL)has a powerful learning and decision-making ability,and the three-dimensional bin packing method based on DRL(DRL-3DBP)has become one of the research hotspots in the field of intelligent logistics.The existing DRL-3DBPs have difficulty in striking a balance between the action space,computational complexity,and exploration capability when solving 3D-BPP with large-size bins.To this end,this paper proposes a four directional cooperative packing(FD-CP)method.The two-stage policy network receives the rotated bin states and generates four directional packing policies.Based on the actions sampled from the four policies,the four states are updated accordingly,and the ac-tion corresponding to the highest value is selected as the packing action.FDCP encourages agent to explore reason-able packing positions in all four directions while compressing the action space and reducing computational com-plexity.Experimental results show that FDCP achieves 1.2％～2.9％improvement in space utilization on the pack-ing problem with 100 × 100 large-sized bin and the numbers of 20,30,and 50 items.
万方数据
维普
