Research on Multimodal Summarization by Integrating Visual and Text Modal Information

With the rapid development of information technology, Internet data is growing exponentially, which makes it difficult for users to extract key information from massive Internet data. Data compression technology represented by text summary has gradually attracted extensive attention from academia and industry. As an extension of text summarization, multimodal summarization task can effectively reduce users’ information burden and improve users’ information acquisition speed by integrating visual and auditory modal information and using the mutual supplement and verification of different modal data. It has high research value in the fields of information retrieval, public opinion analysis, content review and so on. This paper combs the related research of multimodal summarization in recent years, summarizes the existing technologies and related data sets for multimodal summarization tasks, and summarizes the development direction of future research in this field.