Policy improvement in dynamic programming

Policy improvement has a long history and is the essential element in dynamic programming. The general categories of policy improvement can be divided into four aspects including: heuristic methods, approximation methods, sampling methods and numerical improvement. Paralleling with the classic policy improvement methods, several variant tools are also introduced including Lambda Policy, Path Integral, High Confidence in Policy Improvement and Finite Sample Analysis for SARSA in linear function. Moreover, the introductions of those policy improvement methods, evaluation, and comparison between them are illustrated in this paper. There are totally three perspectives where this paper dissects the evaluation from training speed, sampling efficiency and methods ability.