已为您找到该学者10条结果 成果回收站
唐杰, Jie Tang, Hang Li, Yunbo Cao, Zhaohui Tang
,-0001,():
-1年11月30日
Addressed in this paper is the issue of ‘email data cleaning’ for text mining. Many text mining applications need take emails as input. Email data is usually noisy and thus it is necessary to clean it before mining. Several products offer email cleaning features, however, the types of noises that can be eliminated are restricted. Despite the importance of the problem, email cleaning has received little attention in the research community. A thorough and systematic investigation on the issue is thus needed. In this paper, email cleaning is formalized as a problem of non-text filtering and text normalization. In this way, email cleaning becomes independent from any specific text mining processing. A cascaded approach is proposed, which cleans up an email in four passes including non-text filtering, paragraph normalization, sentence normalization, and word normalization. As far as we know, non-text filtering and paragraph normalization have not been investigated previously. Methods for performing the tasks on the basis of Support Vector Machines (SVM) have also been proposed in this paper. Features in the models have been defined. Experimental results indicate that the proposed SVM based methods can significantly outperform the baseline methods for email cleaning. The proposed method has been applied to term extraction, a typical text mining processing. Experimental results show that the accuracy of term extraction can be significantly improved by using the data cleaning method.
Text Mining, Data Cleaning, Email Processing, StatisticalLearning
-
79浏览
-
0点赞
-
0收藏
-
0分享
-
551下载
-
0
-
引用
【期刊论文】A Constraint-Based Probabilistic Framework for Name Disambiguation
唐杰, Duo Zhang, Jie Tang, Juanzi Li, and Kehong Wang
,-0001,():
-1年11月30日
This paper is concerned with the problem of name disambiguation. By name disambiguation, we mean distinguishing persons with the same name. It is a critical problem in many knowledge management applications. Despite much research work has been conducted, the problem is still not resolved and becomes even more serious, in particular with the popularity of Web 2.0. Previously, name disambiguation was often undertaken in either a supervised or unsupervised fashion. This paper first gives a constraint-based probabilistic model for semi-supervised name disambiguation. Specifically, we focus on investigating the problem in an academic researcher social network (http: //arnetminer.org). The framework combines constraints and Euclidean distance learning, and allows the user to refine the disambiguation results. Experimental results on the researcher social network show that the proposed framework significantly outperforms the baseline method using unsupervised hierarchical clustering algorithm.
Name Disambiguation, Social Network Analysis, Digital Library, Semi-supervised Clustering
-
182浏览
-
0点赞
-
0收藏
-
0分享
-
359下载
-
0
-
引用
【期刊论文】Chapter I Information Extraction: Methodologies and Applications
唐杰, Jie Tang, Mingcai Hong, Duo Zhang, Juanzi Li, Bangyong Liang
,-0001,():
-1年11月30日
This chapter is concerned with the methodologies and applications of information extraction. Information is hidden in the large volume of Web pages and thus it is necessary to extract useful information from the Web content, called information extraction. In information extraction, given a sequence of instances, we identify and pull out a subsequence of the input that represents information we are interested in. In the past years, there was a rapid expansion of activities in the information extraction area. Many methods have been proposed for automating the process of extraction. However, due to the heterogeneity and the lack of structure of Web data, automated discovery of targeted or unexpected knowledge information still presents many challenging research problems. In this chapter, we will investigate the problems of information extraction and survey existing methodologies for solving these problems. Several real-world applications of information extraction will be introduced. Emerging challenges will be discussed.
-
39浏览
-
0点赞
-
0收藏
-
0分享
-
285下载
-
0
-
引用
【期刊论文】Arnetminer: expertise oriented search using social networks
唐杰, Juanzi LI, Jie TANG, Jing ZHANG, Qiong LUO, Yunhao LIU, Mingcai HONG
Front. Comput. Sci. China ,-0001,():
-1年11月30日
Expertise Oriented Search (EOS) aims at providing comprehensive expertise analysis on data from distributed sources. It is useful in many application domains, for example, finding experts on a given topic, detecting the confliction of interest between researchers, and assigning reviewers to proposals. In this paper, we present the design and implementation of our expertise oriented search system, Arnetminer (http: //www.arnetminer.net). Arnetminer has gathered and integrated information about a half-million computer science researchers from the Web, including their profiles and publications. Moreover, Arnetminer constructs a social network among these researchers through their co-authorship, and utilizes this network information as well as the individual profiles to facilitate expertise oriented search tasks. In particular, the co-authorship information is used both in ranking the expertise of individual researchers for a given topic and in searching for associations between researchers. We have conducted initial experiments on Arnetminer. Our results demonstrate that the proposed relevancy propagation expert finding method outperforms the method that only uses person local information, and the proposed twostage association search on a large-scale social network is orders of magnitude faster than the baseline method.
social network, expertise search, association search
-
100浏览
-
0点赞
-
0收藏
-
0分享
-
284下载
-
0
-
引用
【期刊论文】A Unified Tagging Approach to Text Normalization
唐杰, Conghui Zhu, Jie Tang, Hang Li, Hwee Tou Ng, Tie-Jun Zhao
,-0001,():
-1年11月30日
This paper addresses the issue of text normalization, an important yet often overlooked problem in natural language processing. By text normalization, we mean converting ‘informally inputted’ text into the canonical form, by eliminating ‘noises’ in the text and detecting paragraph and sentence boundaries in the text. Previously, text normalization issues were often undertaken in an ad-hoc fashion or studied separately. This paper first gives a formalization of the entire problem. It then proposes a unified tagging approach to perform the task using Conditional Random Fields (CRF). The paper shows that with the introduction of a small set of tags, most of the text normalization tasks can be performed within the approach. The accuracy of the proposed method is high, because the subtasks of normalization are interdependent and should be performed together. Experimental results on email data cleaning show that the proposed method significantly outperforms the approach of using cascaded models and that of employing independent models.
-
41浏览
-
0点赞
-
0收藏
-
0分享
-
158下载
-
0
-
引用