Role-based multi-robot warehouse coordination with reinforcement learning assisted task selection

This research investigates a method for coordinating multiple robots in large warehouse fulfillment systems. The problem is modeled as a role-structured decentralized partially observable Semi-Markov decision process, which extends traditional multi-agent frameworks to incorporate roles that can influence macro-actions. These roles have eligibility constraints based on real-time factors like load, capacity, and shelf assignments, with the ability to dynamically switch roles through a quorum mechanism.

This approach avoids central coordination while balancing local demands against available agent resources. A novel HSC-Net architecture is introduced to optimize within this structured action space, using role-specific heuristics combined with spatial encoding and attention-based candidate scoring. The system operates asynchronously, with multi-hop relaying handled without fixed infrastructure or agent rendezvous.

Simulations show significant improvements in throughput, completion time, and congestion balance compared to traditional heuristics and deep Q-learning baselines. The work was supported by Yango University's Talent Project and various research institutes and centers in China.