Deep Reinforcement Learning for Two-Player DouDizhu

Recently, DouZero, an AI system for the game of DouDizhu, has been proposed and made a breakthrough by reaching human level in DouDizhu, one of the most popular imperfect information games in China. In order to verify the effectiveness and versatility of this system, and further compare the performance of Deep Monte-Carlo (DMC) and Deep Q-Network (DQN) algorithms, in this paper we implement and improve DouZero system on two-player DouDizhu, a variant of the classic DouDizhu, where there is no cooperation between the players yet with more hidden information. We first revise DouZero system on this game. We then design filter network based on supervised learning to improve the quality of training data and thus accelerate the training process. To improve the learning efficiency, we also extract the state and action features according to the characteristics of the two-player DouDizhu. In addition, a variety of reward functions are especially designed in terms of this game. The experimental results show that in twoplayer DouDizhu with those improvements both DMC and DQN make satisfactory performances, but different from the existing work, DQN achieves a better performance than DMC.