基于语义本体的网络爬虫模型的研究
首发时间:2008-11-06
摘要:针对传统通用网络信息采集系统自身固有的缺陷,根据本体的相关理论,本文提出了基于语义本体的网络爬虫的相关模型,该模型以本体构建相关领域模型,并构建该领域的所有知识概念集合,并对其进行关键字切割与划分,生成能表达主题的主题关键词集合。结合中国《知网》的相关理论与技术,对传统抓取的网页在语义的角度进行分析,对已抓取的网页页面内容及其中链接的扩展元数据等相关信息进行分词及语法语义等相关处理,获取网页内容关键词集合及超链接的关键词集合,然后分别对获取的网页内容关键词与链接关键词进行与先前由本体生成的主体关键词集合采用语义分析算法进行语义相关性的判定,保存需要的网页,并预测及提取与主题相关的URL,从而提高网络资源信息采集相关率。
For information in English, please click here
Research Of Web Crawler Based On Semantic Ontology
Abstract:According to inborn defects of the traditional general web crawler,besides relevant theories of ontology,we put forward a new model of web crawler which is based on semantic technology. The model make full use of ontology to generate a collection of domain knowledge topics which will be processed in key words and divided into several segments.The model will finally produced a collection containing whole key words related with topics.Also,It will deal with pages and URLs which are fetched by our robot with same procession,analyzing the content,dividing the content and extended data and turning them into key words using the theory and technology supplied by HowNet.In order to to increase the rate of relativity between data and our topics,we utilize the extended metadata and the colletion of key words in content to compute the relativity with our collection of topics generated before,then get the needed pages saved and extract the URLs which are most related with topics from the pages,and repeat the procedures until there is a end.
Keywords: domain ontology web crawler extended metadata HowNet
基金:
论文图表:
引用
No.2548634637612259****
同行评议
共计0人参与
勘误表
基于语义本体的网络爬虫模型的研究
评论
全部评论0/1000