Dezheng Zhang

A Hardware Architecture for Shared Residuals and Simplified Symmetric-PWL-Based GELU

2024

In this paper, we propose an innovative hardware acceleration scheme to optimize key components of Visual Transformer (ViT) and similar models. It features two main innovations. The first is a shared residual mechanism that integrates …

HA-BNN: Hardware-Aware Binary Neural Networks for Efficient Inference

2024

Binary Neural Networks (BNNs) use 1-bit weights and activations, replacing complex matrix multiplications with XNOR and bitcount operations, leading to a notable decrease in memory needs. It is worth highlighting that current BNNs commonly integrate …