Information Retrieval using Elastic Search
Budget: ₹600 – ₹3,000 INR
Obtain a dataset: The first step for you will be to obtain a dataset. Identify a publicy avaible dataset
containing a set of documents to which you can apply your information retrieval knowledge. Provide a
detailed description of the dataset as well as the information on where to find and download it.
• Indexing: Once you have obtained the dataset, upload it with full text to Elasticsearch. . If you run
into problems using the upload script provided, then feel free to use your own approach. If the dataset is
over 10,000 documents you can upload a sample.
• Searching: Once you have indexed the collection you want to be able to search it. You can do that on
the command line but it would be much better to have an interactive system. You could start with Kibana for
that but you are free to use other open source tools for your GUI.
• Building a Test Collection: Imagine you would like to explore what search engine settings are most
suitable for the collection you are indexing to make search as effective as possible. To start with this you
should devise a small test collection that contains a number of queries together with their expected results.
Identify three specific events covered by the collection and then compose two sample queries for each of these
that you might reasonably expect a user to submit to find documents about this event.
• Evaluation: Once you have a test collection you can explore different search engine settings to see
what effect they have on the evaluation results. To do that you need to identify a suitable metric. Use P@5 as
the metric of choice for this assignment. You can then vary different parameters. You could for example
change the pre-processing pipeline by comparing a system that uses stemming with one that does not.
However, this will require you to re-index the collection. Instead I suggest you try different retrieval models
such as Boolean versus TF.IDF.
• Engineering a Complete System: The final system should have control over all the individual
components so that as the final result we have a complete search engine.
containing a set of documents to which you can apply your information retrieval knowledge. Provide a
detailed description of the dataset as well as the information on where to find and download it.
• Indexing: Once you have obtained the dataset, upload it with full text to Elasticsearch. . If you run
into problems using the upload script provided, then feel free to use your own approach. If the dataset is
over 10,000 documents you can upload a sample.
• Searching: Once you have indexed the collection you want to be able to search it. You can do that on
the command line but it would be much better to have an interactive system. You could start with Kibana for
that but you are free to use other open source tools for your GUI.
• Building a Test Collection: Imagine you would like to explore what search engine settings are most
suitable for the collection you are indexing to make search as effective as possible. To start with this you
should devise a small test collection that contains a number of queries together with their expected results.
Identify three specific events covered by the collection and then compose two sample queries for each of these
that you might reasonably expect a user to submit to find documents about this event.
• Evaluation: Once you have a test collection you can explore different search engine settings to see
what effect they have on the evaluation results. To do that you need to identify a suitable metric. Use P@5 as
the metric of choice for this assignment. You can then vary different parameters. You could for example
change the pre-processing pipeline by comparing a system that uses stemming with one that does not.
However, this will require you to re-index the collection. Instead I suggest you try different retrieval models
such as Boolean versus TF.IDF.
• Engineering a Complete System: The final system should have control over all the individual
components so that as the final result we have a complete search engine.