Benchmarking communicative reinforcement learning frameworks on multi-robot cooperative tasks

Abstract

Industry 4.0 warehousing is characterised by autonomous multi-robot collaboration systems (MRSs) along with other technologies such as digital communication capabilities and the Internet of Things. These MRSs need to behave coherently for the efficient completion of the assigned cooperative tasks. Multi-agent reinforcement learning (MARL) frameworks are currently considered state-of-the-art to control the behaviour of autonomous MRSs. These MARL frameworks can be with learnable or predefined communication. Current works lack any worthwhile evaluation of communicative MARL frameworks on multi-robot cooperative tasks. This work empirically evaluates current state-of-the-art seminal learnable communicative MARL frameworks by comparing their performance against non-communicative MARL frameworks on multi-robot coop-erative tasks in the context of Industry 4.0 warehousing with the assumptions of partial observability and reward sparsity. The results demonstrate that communicative MARL frameworks outperform their counterparts by a fair margin in training (average returns between 11 and 6 against 8 and 4 for highest and lowest values respectively) and execution performances (average returns between 1.24 and 0.29 against 0.49 and 0.19 for highest and lowest values respectively). This leads to the conclusion that communicative MARL is better suited to multi-robot cooperative tasks under the above-mentioned assumptions.