Kaiqing Zhang

Discrete Approximate Information States in Partially Observable Environments

2022 · 2022 American Control Conference (ACC)

The notion of approximate information states (AIS) was introduced in [1] as a methodology for learning task-relevant state representations for control in partially observable systems. They proposed particular learning objectives which attempt to reconstruct the …

Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPs

2023 · arXiv (Cornell University)

We study the problem of computing an optimal policy of an infinite-horizon discounted constrained Markov decision process (constrained MDP). Despite the popularity of Lagrangian-based policy search methods used in practice, the oscillation of policy iterates …