TY - JOUR
T1 - A task-oriented approach to information retrieval evaluation
AU - Hersh, William
AU - Pentecost, Jeffrey
AU - Hickam, David
PY - 1996/1
Y1 - 1996/1
N2 - As retrieval systems become more oriented towards end-users, there is an increasing need for improved methods to evaluate their effectiveness. We performed a task-oriented assessment of two MEDLINE searching systems, one which promotes traditional Boolean searching on human-indexed thesaurus terms and the other natural language searching on words in the title, abstract, and indexing terms. Medical students were randomized to one of the two systems and given clinical questions to answer. The students were able to use each system successfully, with no significant differences in questions correctly answered, time taken, relevant articles retrieved, or user satisfaction between the systems. This approach to evaluation was successful in measuring effectiveness of system use and demonstrates that both types of systems can be used equally well with minimal training.
AB - As retrieval systems become more oriented towards end-users, there is an increasing need for improved methods to evaluate their effectiveness. We performed a task-oriented assessment of two MEDLINE searching systems, one which promotes traditional Boolean searching on human-indexed thesaurus terms and the other natural language searching on words in the title, abstract, and indexing terms. Medical students were randomized to one of the two systems and given clinical questions to answer. The students were able to use each system successfully, with no significant differences in questions correctly answered, time taken, relevant articles retrieved, or user satisfaction between the systems. This approach to evaluation was successful in measuring effectiveness of system use and demonstrates that both types of systems can be used equally well with minimal training.
UR - http://www.scopus.com/inward/record.url?scp=0003040889&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0003040889&partnerID=8YFLogxK
U2 - 10.1002/(SICI)1097-4571(199601)47:1<50::AID-ASI5>3.0.CO;2-1
DO - 10.1002/(SICI)1097-4571(199601)47:1<50::AID-ASI5>3.0.CO;2-1
M3 - Article
AN - SCOPUS:0003040889
SN - 0002-8231
VL - 47
SP - 50
EP - 56
JO - Journal of the American Society for Information Science
JF - Journal of the American Society for Information Science
IS - 1
ER -