Home » Publication » 19698

Dettaglio pubblicazione

2019, RoboCup 2019: Robot World Cup XXIII, Pages 44-57 (volume: 11531)

Cooperative Multi-agent Deep Reinforcement Learning in a 2 Versus 2 Free-Kick Task (04b Atto di convegno in volume)

Catacora Ocana J. M., Riccio F., Capobianco R., Nardi D.

In multi-robot reinforcement learning the goal is to enable a team of robots to learn a coordinated behavior from direct interaction with the environment. Here, we provide a comparison of the two main approaches to tackle this challenge, namely independent learners (IL) and joint-action learners (JAL). IL is suitable for highly scalable domains, but it faces non-stationarity issues. Whereas, JAL overcomes non-stationarity and can generate highly coordinated behaviors, but it presents scalability issues due to the increased size of the search space. We implement and evaluate these methods in a new multi-robot cooperative and adversarial soccer scenario, called 2 versus 2 free-kick task, where scalability issues affecting JAL are less relevant given the small number of learners. In this work, we implement and deploy these methodologies on a team of simulated NAO humanoid robots. We describe the implementation details of our scenario and show that both approaches are able to achieve satisfying solutions. Notably, we observe joint-action learners to have a better performance than independent learners in terms of success rate and quality of the learned policies. Finally, we discuss the results and provide conclusions based on our findings.
ISBN: 978-3-030-35698-9; 978-3-030-35699-6
© Università degli Studi di Roma "La Sapienza" - Piazzale Aldo Moro 5, 00185 Roma