Classified Regression for Bayesian Optimization: Robot Learning with\n Unknown Penalties

Learning robot controllers by minimizing a black-box objective cost using\nBayesian optimization (BO) can be time-consuming and challenging. It is very\noften the case that some roll-outs result in failure behaviors, causing\npremature experiment detention. In such cases, the designer is forced to decide\non heuristic cost penalties because the acquired data is often scarce, or not\ncomparable with that of the stable policies. To overcome this, we propose a\nBayesian model that captures exactly what we know about the cost of unstable\ncontrollers prior to data collection: Nothing, except that it should be a\nsomewhat large number. The resulting Bayesian model, approximated with a\nGaussian process, predicts high cost values in regions where failures are\nlikely to occur. In this way, the model guides the BO exploration toward\nregions of stability. We demonstrate the benefits of the proposed model in\nseveral illustrative and statistical synthetic benchmarks, and also in\nexperiments on a real robotic platform. In addition, we propose and\nexperimentally validate a new BO method to account for unknown constraints.\nSuch method is an extension of Max-Value Entropy Search, a recent\ninformation-theoretic method, to solve unconstrained global optimization\nproblems.\n