General Game Playing

The objective of this project is to develop a general game player, as opposed to programs that play specific games, such as Deep Blue (chess) and Chinook (Checkers). This program should be able to take the description (in a language called the "game description language" or GDL) of an arbitrary game and play it competently. Several heuristic approaches have been used to compute the values of positions and moves of unknown games, wihtout the benefit of human interpretation for fine tuning them, as was done in Deep Blue. In contrast, our approach is to train a player, much like TD-Gammon from scratch, using reinforcement learning. However, unlike TD-Gammon, our program cannot use any game-specific information. We envision a transfer learning program that will start with zero knowledge, but with each new game it faces, it will acquire abstract knowledge that may be applicable to (yet) unseen games later on, and then use it to learn effective strategies in such games faster. With an increasingly versatile knowledgebase, we expect the program to be able to play new games with a decreasing need to learn. We thus hope to bootstrap a learning program and prepare it for competition, where it will typically face a game or an opponent only a few times.

We have made some initial strides in this direction, but our program still uses about 3000 repeated matches of a new game to devise a competent strategy for it. Although this number is down from several tens of thousands which would be needed by baseline reinforcement learning if no knowledge transfer was used, it is still impracticably large for the GGP competition. We expect to bring this number down dramatically with deeper transfer, compared to the shallow transfer that we have performed so far.