Gugan Thoppe

Finite Sample Analyses for TD(0) with Function Approximation

2017 · arXiv (Cornell University)

TD(0) is one of the most commonly used algorithms in reinforcement learning. Despite this, there is no existing finite sample analysis for TD(0) with function approximation, even for the linear case. Our work is the …

Finite Sample Analysis of Two-Timescale Stochastic Approximation with Applications to Reinforcement Learning

2017 · arXiv (Cornell University)

Two-timescale Stochastic Approximation (SA) algorithms are widely used in Reinforcement Learning (RL). Their iterates have two parts that are updated using distinct stepsizes. In this work, we develop a novel recipe for their finite sample …

Change Rate Estimation and Optimal Freshness in Web Page Crawling

2020

For providing quick and accurate results, a search engine maintains a local snapshot of the entire web. And, to keep this local cache fresh, it employs a crawler for tracking changes across various web pages. …

Global Convergence Guarantees for Federated Policy Gradient Methods with Adversaries

2024 · arXiv (Cornell University)

Federated Reinforcement Learning (FRL) allows multiple agents to collaboratively build a decision making policy without sharing raw trajectories. However, if a small fraction of these agents are adversarial, it can lead to catastrophic results. We …