Intelligent Positioning Method of Paging Buttons Based on Machine Learning

Information aggregation applications obtain massive amounts of data from the Internet as data support, most of which come from several different websites and need to rely on crawlers to collect from web pages. However, in the process of constructing a crawler system, it is often necessary to manually analyze and locate the web page structure of the target website in order to obtain the most relevant links and traverse the website effectively. In order to improve the automation of the crawler system and reduce manual involvement in the information collection, this paper studies the most common "list page"-"detail page" structure of content display websites, proposes an intelligent positioning method of paging buttons based on machine learning, and conducts experiments on the effectiveness of the method. The experimental results show that the proposed method can intelligently locate the paging buttons in the list page and get the most relevant links, which can improve the automation of the crawler system.