Efficient Hyperparameter Tuning with Dynamic Accuracy Derivative-Free\n Optimization

Many machine learning solutions are framed as optimization problems which\nrely on good hyperparameters. Algorithms for tuning these hyperparameters\nusually assume access to exact solutions to the underlying learning problem,\nwhich is typically not practical. Here, we apply a recent dynamic accuracy\nderivative-free optimization method to hyperparameter tuning, which allows\ninexact evaluations of the learning problem while retaining convergence\nguarantees. We test the method on the problem of learning elastic net weights\nfor a logistic classifier, and demonstrate its robustness and efficiency\ncompared to a fixed accuracy approach. This demonstrates a promising approach\nfor hyperparameter tuning, with both convergence guarantees and practical\nperformance.\n