Sung Ju Hwang

Rethinking Data Augmentation: Self-Supervision and Self-Distillation

2019 · arXiv (Cornell University)

Data augmentation techniques, e.g., flipping or cropping, which systematically enlarge the training dataset by explicitly generating more training samples, are effective in improving the generalization performance of deep neural networks. In the supervised setting, a …

MetaPerturb: Transferable Regularizer for Heterogeneous Tasks and Architectures

2020 · arXiv (Cornell University)

Regularization and transfer learning are two popular techniques to enhance generalization on unseen data, which is a fundamental problem of machine learning. Regularization techniques are versatile, as they are task- and architecture-agnostic, but they do …

Self-Supervised Dataset Distillation for Transfer Learning

2023 · arXiv (Cornell University)

Dataset distillation methods have achieved remarkable success in distilling a large dataset into a small set of representative samples. However, they are not designed to produce a distilled dataset that can be effectively used for …

Co-training and Co-distillation for Quality Improvement and Compression of Language Models

2023 · arXiv (Cornell University)

Knowledge Distillation (KD) compresses computationally expensive pre-trained language models (PLMs) by transferring their knowledge to smaller models, allowing their use in resource-constrained or real-time settings. However, most smaller models fail to surpass the performance of …

ECLIPSE: Efficient Continual Learning in Panoptic Segmentation with Visual Prompt Tuning

2024

Panoptic segmentation, combining semantic and instance segmentation, stands as a cutting-edge computer vision task. Despite recent progress with deep learning models, the dynamic nature of real-world applications necessitates continual learning, where models adapt to new …