一种基于分层策略的移动应用商店主题爬虫的研究与实现
首发时间:2015-12-23
摘要:随着移动互联网的快速发展,移动应用安全检测需求不断增多,对移动应用商店主题爬虫采集能力的要求不断提升。网络爬虫抓取策略是影响主题网络爬虫抓取效率的重要因素。目前针对移动应用商店主题爬虫策略的研究较少,为了提高移动应用商店网络爬虫的采集效率,本文提出了针对主题网站进行结构转换的分层抓取策略,并通过Scrapy网络爬虫框架将该抓取策略实现后证明了该抓取策略的可行性与有效性。
For information in English, please click here
Research and Realization of Mobile App Store Focused Web Crawler Based Hierarchical Policy
Abstract:With the rapid development of mobile Internet, demand of mobile application security testing grow in quantity. The requirements of the mobile app store focused web crawler collection capacity continues to improve. Crawling strategy is an important factor to affect the efficiency of focused web crawler. At present, there are few studies on the crawling strategy for the mobile app store focused crawler. In order to improve the efficiency of the mobile app store focused web crawler, this paper presents a hierarchical fetch strategy base on structural transformation of theme web site. The feasibility and effectiveness of this strategy is demonstrated by the Scrapy web crawler framework.
Keywords: Focused Web Crawler Hierarchical strategy Mobile App Store Scrapy
论文图表:
引用
No.4666054993267144****
同行评议
勘误表
一种基于分层策略的移动应用商店主题爬虫的研究与实现
评论
全部评论0/1000