A task-oriented approach to information retrieval evaluation

William (Bill) Hersh, Jeffrey Pentecost, David Hickam

Research output: Contribution to journalArticle

62 Citations (Scopus)

Abstract

As retrieval systems become more oriented towards end-users, there is an increasing need for improved methods to evaluate their effectiveness. We performed a task-oriented assessment of two MEDLINE searching systems, one which promotes traditional Boolean searching on human-indexed thesaurus terms and the other natural language searching on words in the title, abstract, and indexing terms. Medical students were randomized to one of the two systems and given clinical questions to answer. The students were able to use each system successfully, with no significant differences in questions correctly answered, time taken, relevant articles retrieved, or user satisfaction between the systems. This approach to evaluation was successful in measuring effectiveness of system use and demonstrates that both types of systems can be used equally well with minimal training.

Original languageEnglish (US)
Pages (from-to)50-56
Number of pages7
JournalJournal of the American Society for Information Science
Volume47
Issue number1
StatePublished - Jan 1996

Fingerprint

Information retrieval
information retrieval
Students
Thesauri
evaluation
thesaurus
indexing
Evaluation
medical student
language
student

ASJC Scopus subject areas

  • Engineering(all)

Cite this

A task-oriented approach to information retrieval evaluation. / Hersh, William (Bill); Pentecost, Jeffrey; Hickam, David.

In: Journal of the American Society for Information Science, Vol. 47, No. 1, 01.1996, p. 50-56.

Research output: Contribution to journalArticle

@article{d4c0fa1c06dc4987885ad8f4e9a7cc3a,
title = "A task-oriented approach to information retrieval evaluation",
abstract = "As retrieval systems become more oriented towards end-users, there is an increasing need for improved methods to evaluate their effectiveness. We performed a task-oriented assessment of two MEDLINE searching systems, one which promotes traditional Boolean searching on human-indexed thesaurus terms and the other natural language searching on words in the title, abstract, and indexing terms. Medical students were randomized to one of the two systems and given clinical questions to answer. The students were able to use each system successfully, with no significant differences in questions correctly answered, time taken, relevant articles retrieved, or user satisfaction between the systems. This approach to evaluation was successful in measuring effectiveness of system use and demonstrates that both types of systems can be used equally well with minimal training.",
author = "Hersh, {William (Bill)} and Jeffrey Pentecost and David Hickam",
year = "1996",
month = "1",
language = "English (US)",
volume = "47",
pages = "50--56",
journal = "Journal of the Association for Information Science and Technology",
issn = "2330-1635",
publisher = "John Wiley and Sons Ltd",
number = "1",

}

TY - JOUR

T1 - A task-oriented approach to information retrieval evaluation

AU - Hersh, William (Bill)

AU - Pentecost, Jeffrey

AU - Hickam, David

PY - 1996/1

Y1 - 1996/1

N2 - As retrieval systems become more oriented towards end-users, there is an increasing need for improved methods to evaluate their effectiveness. We performed a task-oriented assessment of two MEDLINE searching systems, one which promotes traditional Boolean searching on human-indexed thesaurus terms and the other natural language searching on words in the title, abstract, and indexing terms. Medical students were randomized to one of the two systems and given clinical questions to answer. The students were able to use each system successfully, with no significant differences in questions correctly answered, time taken, relevant articles retrieved, or user satisfaction between the systems. This approach to evaluation was successful in measuring effectiveness of system use and demonstrates that both types of systems can be used equally well with minimal training.

AB - As retrieval systems become more oriented towards end-users, there is an increasing need for improved methods to evaluate their effectiveness. We performed a task-oriented assessment of two MEDLINE searching systems, one which promotes traditional Boolean searching on human-indexed thesaurus terms and the other natural language searching on words in the title, abstract, and indexing terms. Medical students were randomized to one of the two systems and given clinical questions to answer. The students were able to use each system successfully, with no significant differences in questions correctly answered, time taken, relevant articles retrieved, or user satisfaction between the systems. This approach to evaluation was successful in measuring effectiveness of system use and demonstrates that both types of systems can be used equally well with minimal training.

UR - http://www.scopus.com/inward/record.url?scp=0003040889&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0003040889&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:0003040889

VL - 47

SP - 50

EP - 56

JO - Journal of the Association for Information Science and Technology

JF - Journal of the Association for Information Science and Technology

SN - 2330-1635

IS - 1

ER -