Malvina Nikandrou

Shaking Up VLMs: Comparing Transformers and Structured State Space Models for Vision & Language Modeling

2024

This study explores replacing Transformers in Visual Language Models (VLMs) with Mamba, a recent structured state space model (SSM) that demonstrates promising performance in sequence modeling.We test models up to 3B parameters under controlled conditions, …