Yaodong Li

Optimal Bandwidth Allocation for Web Crawler Systems

2019

Web crawler is an important tool to obtain the information from the Internet in time. In a typical web crawler system with the limited bandwidth, there are many websites required to be crawled with different …