Replay Overshooting: Learning Stochastic Latent Dynamics with the Extended Kalman Filter

This paper presents replay overshooting (RO), an algorithm that uses properties of the extended Kalman filter (EKF) to learn nonlinear stochastic latent dynamics models suitable for long-horizon prediction. We build upon overshooting methods used to train other prediction models and recover a novel variational learning objective. Further, we use RO to extend another objective that acts as a surrogate for the true log-likelihood, and show that this objective empirically yields better models than the variational one. We evaluate RO on two tasks: prediction of synthetic video frames of a swinging motorized pendulum and prediction of the planar position of various objects being pushed by a real manipulator (MIT Push Dataset). Our model outperforms several other prediction models on both quantitative and qualitative metrics.