Khe Chai Sim

Profit: Benchmarking Personalization and Robustness Trade-off in Federated Prompt Tuning

2023 · arXiv (Cornell University)

In many applications of federated learning (FL), clients desire models that are personalized using their local data, yet are also robust in the sense that they retain general global knowledge. However, the presence of data …

TransformerFAM: Feedback attention is working memory

2024 · arXiv (Cornell University)

While Transformers have revolutionized deep learning, their quadratic attention complexity hinders their ability to process infinitely long inputs. We propose Feedback Attention Memory (FAM), a novel Transformer architecture that leverages a feedback loop to enable …