Towards new measures of information retrieval evaluation

William R. Hersh, Diane L. Elliot, David H. Hickam, Stephanie L. Wolf, Anna Molnar

Research output: Contribution to journalConference articlepeer-review

27 Scopus citations

Abstract

All of the methods currently used to evaluate information retrieval (IR) systems have limitations in their ability to measure how well users are able to acquire information. We utilized an approach to assessing information obtained based on the user's ability to answer questions from a short-answer test. Senior medical students took the ten-question test and then searched one of two IR systems on the five questions for which they were least certain of their answer. Our results showed that pre-searching scores on the test were low but that searching yielded a high proportion of answers with both systems. These methods are able to measure information obtained, and will be used in subsequent studies to assess differences among IR systems.

Original languageEnglish (US)
Pages (from-to)164-170
Number of pages7
JournalSIGIR Forum (ACM Special Interest Group on Information Retrieval)
DOIs
StatePublished - 1995
EventProceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval - Seattle, WA, USA
Duration: Jul 9 1995Jul 13 1995

ASJC Scopus subject areas

  • Management Information Systems
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Towards new measures of information retrieval evaluation'. Together they form a unique fingerprint.

Cite this