Dongyoon Han

Masking meets Supervision: A Strong Learning Alliance

2023 · arXiv (Cornell University)

Pre-training with random masked inputs has emerged as a novel trend in self-supervised training. However, supervised learning still faces a challenge in adopting masking augmentations, primarily due to unstable training. In this paper, we propose …

NegMerge: Sign-Consensual Weight Merging for Machine Unlearning

2024 · arXiv (Cornell University)

Machine unlearning aims to selectively remove specific knowledge from a trained model. Existing approaches, such as Task Arithmetic, fine-tune the model on the forget set to create a task vector (i.e., a direction in weight …