TY - JOUR
T1 - Relevance and retrieval evaluation
T2 - Perspectives from medicine
AU - Hersh, William
PY - 1994/4
Y1 - 1994/4
N2 - The traditional notion of topical relevance has allowed much useful work to be done in the evaluation of retrieval systems, but has limitations for complete assessment of retrieval systems. While topical relevance can be effective in evaluating various indexing and retrieval approaches, it is ineffective for measuring the impact that systems have on users. An alternative is to use a more situational definition of relevance, which takes account of the impact of the system on the user. Both types of relevance are examined from the standpoint of the medical domain, concluding that each have their appropriate use. But in medicine there is increasing emphasis on outcomes‐oriented research which, when applied to information science, requires that the impact of an information system on the activities which prompt its use be assessed. An iterative model of retrieval evaluation is proposed, starting first with the use of topical relevance to insure documents on the subject can be retrieved. This is followed by the use of situational relevance to show the user can interact positively with the system. The final step is to study how the system impacts the user in the purpose for which the system was consulted, which can be done by methods such as protocol analysis and simulation. These diverse types of studies are necessary to increase our understanding of the nature of retrieval systems. © 1994 John Wiley & Sons, Inc.
AB - The traditional notion of topical relevance has allowed much useful work to be done in the evaluation of retrieval systems, but has limitations for complete assessment of retrieval systems. While topical relevance can be effective in evaluating various indexing and retrieval approaches, it is ineffective for measuring the impact that systems have on users. An alternative is to use a more situational definition of relevance, which takes account of the impact of the system on the user. Both types of relevance are examined from the standpoint of the medical domain, concluding that each have their appropriate use. But in medicine there is increasing emphasis on outcomes‐oriented research which, when applied to information science, requires that the impact of an information system on the activities which prompt its use be assessed. An iterative model of retrieval evaluation is proposed, starting first with the use of topical relevance to insure documents on the subject can be retrieved. This is followed by the use of situational relevance to show the user can interact positively with the system. The final step is to study how the system impacts the user in the purpose for which the system was consulted, which can be done by methods such as protocol analysis and simulation. These diverse types of studies are necessary to increase our understanding of the nature of retrieval systems. © 1994 John Wiley & Sons, Inc.
UR - http://www.scopus.com/inward/record.url?scp=84989554840&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84989554840&partnerID=8YFLogxK
U2 - 10.1002/(SICI)1097-4571(199404)45:3<201::AID-ASI9>3.0.CO;2-W
DO - 10.1002/(SICI)1097-4571(199404)45:3<201::AID-ASI9>3.0.CO;2-W
M3 - Article
AN - SCOPUS:84989554840
SN - 0002-8231
VL - 45
SP - 201
EP - 206
JO - Journal of the American Society for Information Science
JF - Journal of the American Society for Information Science
IS - 3
ER -