دانلود رایگان ترجمه مقاله استخراج کلید واژه با استفاده از زنجیرهای واژگانی – الزویر ۲۰۰۷

استفاده از زنجیره های واژگانی برای استخراج لغات کلیدی

Using lexical chains for keyword extraction

Abstract

Keywords can be considered as condensed versions of documents and short forms of their summaries. In this paper, the problem of automatic extraction of keywords from documents is treated as a supervised learning task. A lexical chain holds a set of semantically related words of a text and it can be said that a lexical chain represents the semantic content of a portion of the text. Although lexical chains have been extensively used in text summarization, their usage for keyword extraction problem has not been fully investigated. In this paper, a keyword extraction technique that uses lexical chains is described, and encouraging results are obtained.

۱ Introduction

Keywords can be considered as brief summaries of a text. Therefore it is possible to think of them as a set of phrases semantically covering most of the text. Although a summary of a text is capable of providing more information about the text than keywords of the text, the summary may not be suitable for some applications due to the complex structure of sentences. Keywords are not replacements for summarization but alternative summary representations that could be consumed by other applications more easily. Since they are concise representations of the underlying text, it is possible to use them in different applications such as indexing in search engines or text categorization. Keywords enable readers to decide whether a document is relevant for them or not. They can also be used as low cost measures of similarity between documents. Unfortunately, a great portion of existing documents available today does not have keywords available for them. Considering the fact that it is a hard and time consuming task to assign keywords to documents, it is desirable to automate this task by machine learning and natural language processing (NLP) techniques. Authors can assign keyphrases for their documents, and those keyphrases might or might not occur in the text. In automatic keyphrase extraction, most indicative phrases in a document are selected as keyphrases for that document. Thus, automatic keyphrase extraction algorithms are limited with phrases that appear in the text. More general form of keyphrase extraction is keyphrase generation which does not select phrases from the document, but generates and assigns keyphrases for the document. In this paper, we concentrate on ‘‘keywords’’ instead of ‘‘keyphrases’’ to emphasize the fact that keyphrases can be composed of more than one word, and we only extract keywords. We believe that a keyword of a text should be semantically related with the words of the text. A lexical chain for a text contains a subset of the words (word senses) in the text. The words in the lexical chain are semantically related. A lexical chain may cover a small or big portion of the text. The number of words and the number of semantic relations among the words can be different for each lexical chain. The coverage and size of a lexical chain can indicate how well the lexical chain represents the semantic content of the text. So, we believe that a keyword which represents the semantic content of the text should be selected from the words of a lexical chain which represents the most of the semantic content of the text. In this paper, we present a keyword extraction method such that it uses the features based on lexical chains in the selection of keywords for a text. Keyword extraction is highly related to automated text summarization. In text summarization, most indicative sentences are extracted to represent the text. In keyword extraction, most indicative keywords are extracted to represent the text. In both of these problems, the features like word frequencies, cue phrases, position in text, lexical chains and discourse structure are exploited to discover a pattern representing importance in a text. In this paper, we aim to explore the effect of lexical chains in keyword extraction, when the problem is treated as a supervised machine learning task. This learning task uses features based on the lexical chains of words. Since we can build lexical chains for words only (not for phrases) using the WordNet ontology (Fellbaum, 1998), we concentrate on the keyword extraction problem instead of keyphrase extraction. Although we have experimented with different classifiers such as Naive Bayes, we obtained better results with the decision tree induction algorithm C4.5 (Quinlan, 1993). For this reason, we have used C4.5 in order to represent the keyword extraction problem as a learning task. We used C4.5 with two different sets of features. In our baseline system, we used only the text features (without using any feature based on the lexical chains of words). In the second case, C4.5 was used with the features based on the lexical chains in addition to the features used in the baseline system. Then we compare the results of these two versions. We have obtained better results when the features based on the lexical chains were used. We first present the related work on keyword extraction and lexical chains in Section 2. Then lexical chains and creation of lexical chains are described in Section 3. After lexical chain based features that are used in our keyword extraction system are explained in Section 4, we discuss the results of our keyword extraction method in Section 5. Finally, we give some concluding remarks in Section 6.