sit772 database and information retrieval · 2018. 9. 10. · sit772 database and information...
TRANSCRIPT
-
SIT772 Database and Information Retrieval
Assessment Task 2 Trimester 2 2018
Deakin University 1 FutureLearn
This task evaluates the student's technical skills in the management of unstructured data, with potential usage in real applications.
This assessment supports student understandings of the techniques related to unstructured data management and data processing.
This assessment task enables students to demonstrate their proficiency against Unit Learning Outcome 5.
ULO5: Demonstrate data retrieval skills in the context of a data processing system.
Assessment 2 (Individual) Written report
Weight (% of total mark) 30
Due date Friday, 21 September 2018 11:59 pm (AEST).
Submission method
Referencing style
Please read the rubric carefully as it outlines what criteria your assessment will be evaluated on.
Instructions
• Read these instructions
• Answer as many questions as possible
• Place your name, ID and answers in your document.
• Please submit your word file with your answers and graphs (embedded) where appropriateas a SINGLE document in the Submission Portal.
Do not submit PDF files
Through CloudDeakin via FutureLearn
Harvard
-
Deakin University 2 FutureLearn
Question 1 (15 marks):
Suppose you have joined a search engine development team to design a search algorithm based on both the Vector model and the Boolean model.
You are supposed to collect unstructured documents for the following topics, and apply an index technique to convert them into an inverted index.
Please collect 3 documents (less than 30 words for each) in three different topics. Topics are listed as follows, you can also choose some other topics you prefer.
• Science
• Computer Vision
• Search Engine
• Database
• Security and privacy.
An example of document:
“Google is the most widely used Web search engine in the World. It claims to be the World’s most comprehensive search engine, indexing over 2.4 billion Web pages.”
1. Creating the inverted index. In the process of creating the inverted index, pleasecomplete the following steps:
a. Find a stopword list in the Internet and remove all stopwords and punctuationfrom those three documents. Then apply Porter’s stemming algorithm to alldocuments. Note that there are plenty of online stemming applications available,and you may use Porter algorithm for this question. The output will be a set ofstemmed terms.
b. Create a merged inverted list including the within-document frequencies for eachterm.
c. Use the index created in step (b) to create a dictionary and the related posting file.
d. You may like to test the inverted index by using some keywords, please selectsome keywords from the documents. For example: google, web, search.
2. Boolean and Vector queries.
a. Please design three Boolean queries, (for example, web AND search) and list therelevant documents for each query.
b. Please use the Vector model to query on the inverted index, and compare theresult with the Boolean model. (Hint: you can use cosine similarity and set asimilarity threshold).
-
SIT772 Database and Information Retrieval
Assessment Task 2 Trimester 2 2018
Deakin University 3 FutureLearn
Question 2 (IR Evaluation, 15 marks):
For this exercise, you are required to evaluate the performance of different search engines.
First, please find two search engines you are familiar with, such as Google, Bing, Yahoo!, etc.
Second, please choose a target in the following groups, and design two queries to search in both search engines.
The target is chosen by the last number of your student ID. For example, if your student ID ends with the number is 1, please choose target 1; if it is 0, please choose target 10.
• Target 1: obtain the unit guide of SIT771.
• Target 2: obtain the unit guide of SIT772.
• Target 3: obtain the unit guide of SIT773.
• Target 4: obtain the unit guide of SIT774.
• Target 5: obtain the price of the new Macbook.
• Target 6: obtain the price of the new iPhone.
• Target 7: obtain the price of a Lenovo Laptop.
• Target 8: obtain the install document of MongoDB.
• Target 9: obtain the manual of MongoDB.
• Target 10: obtain the operation guide of MongoDB.
Select the first 20 results in both search engines, if they return the target, then mark them as relevant documents, otherwise, they are irrelevant. The following exercises are based on your search results.
a. List your target and designed search queries (you can use any keywords you think are related to the target). For Search Engine 1, plot the precision versus recall curves for Query 1 and Query 2, interpolated to the 11 standard recall levels. Also plot the average precision versus recall curve for Search Engine 1 (all three curves should be on a single chart).
b. For Search Engine 2, plot the precision versus recall curves for Query 1 and Query 2, interpolated to the 11 standard recall levels. Also plot the average precision versus recall curve for Search Engine 2 (all three curves should be on a single chart, but a separate chart from that used in part (a)).
c. Plot the averages for Search Engine 1 and Search Engine 2 on a separate chart, and compare the algorithms in terms of precision and recall. Which search engine do you think is superior? Why?
-
SIT772 DATABASE AND INFORMATION RETRIEVAL, Trimester2 2018 Assessment Task 2 Rubric
1
Question 1 Excellent Good Fair Unsatisfactory
Output porter stemming results
3 marks
Output suitable word list on all of the three documents.
2 marks
Output suitable word list on two of the three documents.
1 mark
Output suitable word list on one of the documents.
0 mark
Fail to output stemming results on any document.
Create a merged inverted list including the within-document frequencies list index
3 marks
Successfully created the merged inverted list index.
2 marks
2/3 of terms are correctly indexed. 1 mark
Half of the terms are correctly indexed. 0 mark
Fail to create an inverted list index.
Create a dictionary and the related posting file
3 marks
Successfully created the dictionary. All terms are pointing to the posting file
2 marks
2/3 of the dictionary file is correct. All related terms are pointing to the posting file
1 mark
Half of the dictionary file is correct. All related terms are pointing to the posting file
0 mark
Fail to create a dictionary and the related posting file.
Boolean model
3 marks
The correct documents for all of the three queries are correct.
2 marks
The correct documents for two of the three queries are correct.
1 mark
The correct documents for one of the three queries are correct.
0 mark
Fail to get the correct documents for any of the queries.
Vector Model
3 marks
The correct documents for all of the three queries are correct. Provide compare result.
2 marks
The correct documents for two of the three queries are correct. Provide the compare result
1 mark
The correct documents for one of the three queries are correct. Provide the compare result
0 mark
Fail to get the correct documents for any of the queries. Fail to compare vector and Boolean models
Question 2 Excellent Good Fair Unsatisfactory
List the target and related queries
3 marks
List target and both queries. 2 marks
List target, but only one query 1 mark
List target, but not queries 0 mark
Fail to list the target and queries
Search engine 1: Plot recall and precision curve
3 marks
Both Precision and recall values and curves are correct.
2 marks
Precision and recall values for both queries are correct, but related curves are incorrect.
1 mark
Precision and recall values for one query are correct, but another one is incorrect.
0 mark
Curves are missing or wrong.
Search engine 2: Plot recall and precision curve
3 marks
Both values and curves are correct. 2 marks
Precision and recall values for both queries are correct, but related curves are incorrect.
1 mark
Precision and recall values for one query are correct, but another one is incorrect.
0 mark
Curves are missing or wrong.
Interpolation of both curves of search engine 1 and 2.
3 marks
Provide correct interpolations for both curves.
2 marks
Provide approximate correct interpolation but with some key points missing
1 mark
Provide one correct interpolation but other’s a wrong
0 mark
Fail to provide the interpolation
Compare two engines 3 marks
Obtain correct comparison result with full explanation.
2 marks
Obtain correct comparison result, but does not provide enough explanation.
1 mark
Compare part of result, but the result in doubtable
0 mark
Fail to compare them
Overall score Level 4 24 or more
Level 3 16 or more
Level 2 8 or more
Level 1 0 or more
SIT772-Assessment task 2 T2 2018.pdfInstructionsDo not submit PDF filesQuestion 1 (15 marks):Question 2 (IR Evaluation, 15 marks):
SIT772-Assessment-2-rubric T2 2018.pdf