Advanced Data Structure Project

Only available on StudyMode
  • Topic: Document retrieval, Index, Vector space model
  • Pages : 4 (1003 words )
  • Download(s) : 161
  • Published : November 26, 2012
Open Document
Text Preview
CSCI4117
Advanced Data Structure Project Proposal

Yejia Tong/B00537881
2012.11.5

1. Title of Project

Succinct data structure in top-k documents retrieval

2. Objective of Research

The main aim of this project is to discover how to efficiently find the k documents where a given pattern occurs most frequently. While the problem has been discussed in many papers and solved in various ways, our research is to look for the novel algorithms and (succinct) data structures among lately related materials and find the one dominating almost all the space/time tradeoff.

3. Background/History of the Study

Before we beigin our aim to find a such a succinct data structure, there are a number of fundamental works in our approach. There exist two main among many ideas in classic information retrieval: inverted index and term frequency. (Angelos, Giannis, Epimeneidis, Euripides, & Evangelos, 2005) The inverted index is a also referred to as postings file, which is an index dara structure storing a mapping from content. It is the most utilized data structure in the Information Retrieval domain, used on a large scale for example in search engines. Term frequency is a measure of how often a term is found in a collection of documents. However, there are restricted assumptions for the efficiency of the ideas: the text must be easily tokenized into words, there must not be too many different words, and queries must be whole words or phrases, causing lots of difficulty in the document retrieval via various languages.

Moreover, one of the attractive properties of an inverted file is that it is easily compressible while still supporting fast queries. In practice, an inverted file occupies space close to that if a compressed document collection. (Niko & Veli, 2007) In further development, people find efficient data structures such as suffix arrays and suffix trees (full-text indexes) providing good space/time efficiency to inverted files. Recently, several...
tracking img