Zhelai Ding

A Hardware Architecture for Shared Residuals and Simplified Symmetric-PWL-Based GELU

2024

In this paper, we propose an innovative hardware acceleration scheme to optimize key components of Visual Transformer (ViT) and similar models. It features two main innovations. The first is a shared residual mechanism that integrates …