Alistair Moffat

Users, Adaptivity, and Bad Abandonment

2018

We consider two recent proposals for effectiveness metrics that have been argued to be adaptive, those of Moffat et al. (ACM TOIS, 2017) and Jiang and Allan (CIKM, 2017), and consider the user interaction models …

A preliminary comparison of job, talent, and web search

2018 · CEUR Workshop Proceedings

Copyright held by the author(s). This paper presents an initial comparison of user behavior in job, talent, and web search using query and click logs from a popular employment marketplace. The observations suggest that the …

Batch Evaluation Metrics in Information Retrieval: Measures, Scales, and Meaning

2022 · arXiv (Cornell University)

A sequence of recent papers has considered the role of measurement scales in information retrieval (IR) experimentation, and presented the argument that (only) uniform-step interval scales should be used, and hence that well-known metrics such …

Can machine translation systems be evaluated by the crowd alone

2015 · Natural Language Engineering

Abstract Crowd-sourced assessments of machine translation quality allow evaluations to be carried out cheaply and on a large scale. It is essential, however, that the crowd's work be filtered to avoid contamination of results through …

Incorporating User Expectations and Behavior into the Measurement of Search Effectiveness

2017 · ACM Transactions on Information Systems

Information retrieval systems aim to help users satisfy information needs. We argue that the goal of the person using the system, and the pattern of behavior that they exhibit as they proceed to attain that …