Song‐Chun Zhu

Modeling and Inferring Human Intents and Latent Functional Objects for Trajectory Prediction

2016 · arXiv (Cornell University)

This paper is about detecting functional objects and inferring human intentions in surveillance videos of public spaces. People in the videos are expected to intentionally take shortest paths toward functional objects subject to obstacles, where …

Mining Interpretable AOG Representations from Convolutional Networks via Active Question Answering

2018 · arXiv (Cornell University)

In this paper, we present a method to mine object-part patterns from conv-layers of a pre-trained convolutional neural network (CNN). The mined object-part patterns are organized by an And-Or graph (AOG). This interpretable AOG representation …

Learning by Fixing: Solving Math Word Problems with Weak Supervision

2020 · arXiv (Cornell University)

Previous neural solvers of math word problems (MWPs) are learned with full supervision and fail to generate diverse solutions. In this paper, we address this issue by introducing a \textit{weakly-supervised} paradigm for learning MWPs. Our …

Iterative Teacher-Aware Learning

2021 · arXiv (Cornell University)

In human pedagogy, teachers and students can interact adaptively to maximize communication efficiency. The teacher adjusts her teaching method for different students, and the student, after getting familiar with the teacher's instruction mechanism, can infer …

Latent Diffusion Energy-Based Model for Interpretable Text Modeling

2022 · arXiv (Cornell University)

Latent space Energy-Based Models (EBMs), also known as energy-based priors, have drawn growing interests in generative modeling. Fueled by its flexibility in the formulation and strong modeling power of the latent space, recent works built …

Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning

2022 · arXiv (Cornell University)

Mathematical reasoning, a core ability of human intelligence, presents unique challenges for machines in abstract thinking and logical reasoning. Recent large pre-trained language models such as GPT-3 have achieved remarkable progress on mathematical reasoning tasks …

LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoning

2024 · arXiv (Cornell University)

Conventional Task and Motion Planning (TAMP) approaches rely on manually crafted interfaces connecting symbolic task planning with continuous motion generation. These domain-specific and labor-intensive modules are limited in addressing emerging tasks in real-world settings. Here, …