Antonín Vobecký

MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments

2023 · arXiv (Cornell University)

Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks for very large fully-annotated datasets. Different classes of self-supervised learning offer representations with either good contextual reasoning properties, e.g., using masked …