SibylOpt: Managing Green Data Centers Using Off-Online Deep Reinforcement Learning

We introduce SibylOpt, a system that applies deep reinforcement learning (DRL) to optimize the operation of a green data center (DC). SibylOpt uses an offline reinforcement learning algorithm, Advantage-Weighted Actor-Critic (AWAC), that requires historical data for training but does not depend on a DC simulator, which is effort intensive to build and keep updated as the DC evolves. SibylOpt then augments the offline training with online learning for refinement and adaptation to changes. We apply SibylOpt to the management of a small green DC that has onsite solar energy generation and a hybrid cooling system that includes "free-cooling." Evaluation results (using simulation) show that the offline trained SibylOpt achieves higher rewards trading off job wait time, cooling, and grid electricity consumption compared to two baseline policies. It is also competitive with PPO, a DRL approach that requires an accurate DC simulator for training.