A. Mitra

A Simple Finite-Time Analysis of TD Learning With Linear Function Approximation

2024 · IEEE Transactions on Automatic Control

We study the finite-time convergence of temporal-difference (TD) learning with linear function approximation under Markovian sampling. Existing proofs for this setting either assume a projection step in the algorithm to simplify the analysis, or require …