Jiaqi Zhang

Identifiability Guarantees for Causal Disentanglement from Purely Observational Data

2024 · arXiv (Cornell University)

Causal disentanglement aims to learn about latent causal factors behind data, holding the promise to augment existing representation learning methods in terms of interpretability and extrapolation. Recent advances establish identifiability results assuming that interventions on …

GradPower: Powering Gradients for Faster Language Model Pre-Training

2025 · ArXiv.org

We propose GradPower, a lightweight gradient-transformation technique for accelerating language model pre-training. Given a gradient vector $g=(g_i)_i$, GradPower first applies the elementwise sign-power transformation: $φ_p(g)=({\rm sign}(g_i)|g_i|^p)_{i}$ for a fixed $p>0$, and then feeds the transformed …