Rong Xiao

Not All Attention Heads Are What You Need: Refining CLIP's Image Representation with Attention Ablation

2025 · arXiv (Cornell University)

This paper investigates the role of attention heads in CLIP's image encoder. Building on interpretability studies, we conduct an exhaustive analysis and find that certain heads, distributed across layers, are detrimental to the resulting representations. …

CATN

2020

In a large recommender system, the products (or items) could be in many different categories or domains. Given two relevant domains (e.g., Book and Movie), users may have interactions with items in one domain but …