Blog

The latest from Google Research

Introducing Google Research Football: A Novel Reinforcement Learning Environment

viernes, 7 de junio de 2019

Posted by Karol Kurach, Research Lead and Olivier Bachem, Research Scientist, Google Research, Zürichreinforcement learningroboticsself-driving carsmoreAtari console gamesgame of GoDota 2Starcraft 2Google Research Football Environmentopen-source code on Github Football Engine Gameplay Football

The Football Engine is an advanced football simulation that supports all the major football rules such as kickoffs (top left), goals (top right), fouls, cards (bottom left), corner and penalty kicks (bottom right), and offside.

stochasticOpenAI GymFootball Benchmarksrule-basedDQNIMPALAsample-efficient RLsparse rewardsmodel based RL

The average goal difference of agent versus opponent at different difficulty levels for different baselines. The Easy opponent can be beaten by a DQN agent trained for 20 million steps, while the Medium and Hard opponents require a distributed algorithm such as IMPALA that is trained for 200 million steps.

Football Academy & Future Directionscurriculum learning

Top: A successful policy that runs towards the goal (as required, since a number of opponents chase our player) and scores against the goal-keeper. Second: A beautiful way to drive and finish a counter-attack. Third: A simple way to solve a 2-vs-1 play. Bottom: The agent scores after a corner kick.

AcknowledgmentsThis project was undertaken together with Anton Raichuk, Piotr Stańczyk, Michał Zając, Lasse Espeholt, Carlos Riquelme, Damien Vincent‎, Marcin Michalski, Olivier Bousquet‎ and Sylvain Gelly at Google Research, Zürich. We also wish to thank Lucas Beyer, Nal Kalchbrenner, Tim Salimans and the rest of the Google Brain team for helpful discussions, comments, technical help and code contributions. Finally, we would like to thank Bastiaan Konings Schuiling, who authored and open-sourced the original version of this game.