Affordable Access

Publisher Website

Evaluation measures for interactive information retrieval

Authors
Journal
Information Processing & Management
0306-4573
Publisher
Elsevier
Publication Date
Volume
28
Issue
4
Identifiers
DOI: 10.1016/0306-4573(92)90007-m

Abstract

Abstract Several criteria and measures have been proposed and used in evaluating interactive IR performance. There is no agreement about what is a successful IR performance or which are the best existing evaluation measure(s). This study aims to identify the best evaluation measure(s) for interactive IR performance. Twenty measures of IR performance were selected for study in the natural IR environment, involving 40 real end-users from an academic setting with 40 real information problems, interacting with six professional intermediaries searching in large operational IR systems. These end-users were responsible for the costs of their own searches. This study showed that value of search results as a whole is the best single measure of interactive IR performance among the measures selected. Precision, one of the most important traditional measures of effectiveness, is not significantly correlated with success. Users appear to be more concerned with absolute recall than with precision. The study also identified the two more basic factors for future IR evaluation which can account for a much higher proportion of the total variance than value of search results as a whole alone can. Seventeen new success categories were also suggested for future investigation.

There are no comments yet on this publication. Be the first to share your thoughts.

Statistics

Seen <100 times
0 Comments

More articles like this

Evaluation problems in interactive information ret...

on Information Storage and Retrie... Jan 01, 1970

Towards new measures of information retrieval eval...

on Proceedings / the ... Annual S... 1994

On the role of user-centred evaluation in the adva...

on Information Processing & Manag... Jan 01, 2008
More articles like this..