Jinghan Yao

Flover: A Temporal Fusion Framework for Efficient Autoregressive Model Parallel Inference

2023

Autoregressive models, despite their commendable performance in a myriad of generative tasks, face challenges stemming from their inherently sequential structure. Inference on these models, by design, harnesses a temporal dependency, where the current token's probability …