Jianlei Yang

RoSearch: Search for Robust Student Architectures When Distilling Pre-trained Language Models

2021 · arXiv (Cornell University)

Pre-trained language models achieve outstanding performance in NLP tasks. Various knowledge distillation methods have been proposed to reduce the heavy computation and storage requirements of pre-trained language models. However, from our observations, student models acquired …

GNNavigator: Towards Adaptive Training of Graph Neural Networks via Automatic Guideline Exploration

2024

Graph Neural Networks (GNNs) succeed significantly in many applications recently. However, balancing GNNs training runtime cost, memory consumption, and attainable accuracy for various applications is non-trivial. Previous training methodologies suffer from inferior adaptability and lack …