Chongyang Bai

UIBert: Learning Generic Multimodal Representations for UI Understanding

2021 · arXiv (Cornell University)

To improve the accessibility of smart devices and to simplify their usage, building models which understand user interfaces (UIs) and assist users to complete their tasks is critical. However, unique challenges are proposed by UI-specific …